v0.6.0

What's Changed

• Update openrouter example by @onur-askui in #71
• fix(PynputAgentOs): Fix missing run_command by @programminx-askui in #70
• fix: Change private variable _tools to public tools for accessibility by @onur-askui in #73
• docs(agent): fix type annotation of VisionAgent.act() by @adi-wan-askui in #74
• docs(agent): update docs + tests with regards to nested/recursive response schemas by @adi-wan-askui in #72

🚀 Features

• Use recursive response schemas with VisionAgent.get()

🐞 Bug Fixes

• Use nested response schemas with VisionAgent.get()
• Response type of VisionAgent.get() was ResponseSchemaBase instead of class extending ResponseSchemaBase when passing that class with the response_schema arg

🚨 Breaking Changes

• Renamed VisionAgent._tools to VisionAgent.tools as tools should be public

Full Changelog: v0.5.3...v0.5.4

v0.5.3

What's Changed

• Add "switch" as element class locator by @adi-wan-askui in #66
• Open up underlying messages of VisionAgent.act() and extract api from chat (experimental) by @adi-wan-askui in #64
• OpenRouter Integration by @onur-askui in #67

🚀 Features

• Add OpenRouter integration
• Allow passing messages and callbacks to VisionAgent.act()
• Add "switch" element class locator

🐞 Bug Fixes

• Fix logging and reporting of act()
• Fix askui.tools.anthropic.computer.ComputerTool, e.g., hard coding of height and width of screen and that height and width were not set when no screenshot was taken before clicking, mouse moving etc. raising an exception

Other

• Update README to include OpenRouter integration and usage examples
• Improve docstrings for new features
• Structure deps better, e.g., making pynput optional
• Add cursor rules

🚨 Breaking Changes

• ActModel.act() signature changed, original goal parameter is now in messages[0]["content"]
• Rename AgentOs.set_display's displayNumber parameter to display (snake_case, consistent with AskUiControllerClient constructor parameter)
• Rename AskUiControllerClient.set_display's displayNumber parameter to display (snake_case, consistent with AskUiControllerClient constructor parameter)
• Rename AgentOs.mouse to AgentOs.mouse_move (including implementations PynputAgentOs and AskUiControllerClient)
• Remove unused image API in chat

Full Changelog: v0.5.2...v0.5.3

v0.5.2

Rerelease of v0.5.1 as something went wrong releasing.

Full Changelog: v0.5.1...v0.5.2

v0.5.1

🐞 Bug Fixes

• Locate retry to include screenshot by @onur-askui in (#65)

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

• Retry on ElementNotFoundError by @onur-askui in (#54)
• Resolve issues where OS agent fails to stop on close by @mlikasam-askui and @onur-askui in (#58)
• Introduces mouse_up and mouse_down to VisionAgent by @onur-askui in (#60)
• Run cli commands via agent os by @onur-askui and @adi-wan-askui in (#61)
• Processing UI-TARS model coordinate outputs by @zenjieli in (#56)
• Report AskUI computer use by @onur-askui in (#57)
• Register your own models by @adi-wan-askui in (#62)
• Set mause move delay to 500ms by @onur-askui in (#63)

🚀 Features

• New models parameter in VisionAgent constructor
• Retry mechanism into VisionAgent for locate method
• Add mouse_up and mouse_down methods to VisionAgent
• Forward cli command to agentos
• Introduce abstract retry and move default retry to ConfigurableRetry class

🐞 Bug Fixes

• Typing & linting errors
• Some typing issues (mypy) across codebase
• Some examples in docstrings
• Update reporter message from "Anthropic Computer Use" to "AskUI Computer Use"
• Set mouse move delay to 500ms
• Resolve issue where OS agent fails to stop on close
• Fix invalid variable access
• Fix code example of using custom models
• Add validation to ConfigurableRetry

Other

• Add integration tests for custom models
• Make examples in docs clearer
• Add Retry utility class for configurable retry mechanisms
• Add return types in ConfigurableRetry
• Add debug message to _locate
• Rename timeout parameter and remove telemetry for run_command

🚨 Breaking Changes

• Changed order of arguments of VisionAgent.get() to fix typing errors (image moved to last position)
• model_router parameter removed from VisionAgent constructor --> replaced by models parameter
• No default routing to Anthropic models as a fallback to AskUI models (routing only to "askui" if model parameter not set)
• Raising exceptions.ModelNotFoundError and exceptions.ModelTypeMismatchError that may be raised by
  • VisionAgent.act()
  • VisionAgent.click()
  • VisionAgent.get()
  • VisionAgent.locate()
  • VisionAgent.mouse_move()

Full Changelog: v0.4.7...v0.5.0

v0.4.7

Rerelease of v0.4.6 as something went wrong releasing.

Full Changelog: v0.4.6...v0.4.7

v0.4.6

What's Changed

• feat: retry act on 529 and 429 errors by @onur-askui in #55

Full Changelog: v0.4.5...v0.4.6

v0.4.5

What's Changed

• increase the default timeout of act request to AskUI api to increase robustness

Full Changelog: v0.4.4...v0.4.5

v0.4.4

What's Changed

• Cl 1388 act anthropic via askui by @onur-askui in #53

New Contributors

• @onur-askui made their first contribution in #53

Full Changelog: v0.4.3...v0.4.4

v0.4.3

🐞 Bug Fixes

• regression: rename AgentToolbox.agent_os back to AgentToolbox.os

Full Changelog: v0.4.2...v0.4.3