v0.22.9

Release Notes: v0.22.9

🚀 New Features

Android Agent

• Added device parameter to AndroidVisionAgent constructor for device selection by serial number or index
• Added act_tools parameter to AndroidVisionAgent (matching VisionAgent functionality)

Reporting

• Added theme toggle (light/dark) to HTML reports
• Updated HTML report styling with CSS variables and improved color scheme
• Added reporting messages for key_combination(), shell(), drag_and_drop(), and swipe() methods
• Moved reporting from AndroidAgentOsFacade to PpadbAgentOs to eliminate duplicate reporting

🐛 Bug Fixes

Android Agent

• Added UnknownAndroidDisplay class for handling cases where display information cannot be determined

---

Full Changelog: v0.22.8...v0.22.9

v0.22.8

What's Changed

• fix inference of correct computer use beta flag by @philipph-askui in #204
• Add Caching to VisionAgent by @philipph-askui in #201

Full Changelog: v0.22.7...v0.22.8

v0.22.7

What's Changed

• Automatic inference of correct Computer-use Flag by @philipph-askui in #195
• Introduce otel tracing by @onur-askui in #196
• feat(chat): add parent_id to messages for tree navigation by @danyalxahid-askui in #186

🚀 Features

• OpenTelemetry Tracing Docs
• Introduce branching for conversations. Support rerunning and editing user messages.

🐛 Bug Fixes

• Use computer-use-2025-11-24 beta flag for claude-opus-4-5-20251101

Full Changelog: v0.22.6...v0.22.7

v0.22.6

What's Changed

• feat: enhance Agent Android tap tool to allow N taps in a row with optional delay by @mlikasam-askui in #199

🚀 Features

• Enhanced Agent Tap Actions
  • The Android Vision Agent can now perform multiple consecutive taps on Android devices with a configurable delay between taps, enabling more complex instructions.
    • Supports commands like:

      agent.act("increase the temperature in the climate control by 20 degrees with 0 milliseconds delay between taps")

    • Handles multi-tap instructions precisely, useful for automated agent-driven tasks
    • Delay between taps can be specified in milliseconds

---

Full Changelog: v0.22.5...v0.22.6

v0.22.5

What's Changed

• feat: add allure reporter by @programminx-askui in #191
• fix: android multi-screen support by @mlikasam-askui in #197

🚀 Features

• Allure Test Reporting Integration

  • AllureReporter: Seamlessly integrate Vision Agent test results into the Allure reporting framework.
    • Records each agent interaction as a structured Allure test step
    • Automatically attaches screenshots to improve debugging and test insights
    • Performs eager dependency checking — if Allure is not installed, an ImportError is raised during initialization

  from askui import VisionAgent
  from askui.reporting import AllureReporter
  
  with VisionAgent(reporter=[AllureReporter()]) as agent:
      agent.act("Click the login button")
      # Actions become Allure steps with screenshots in the report

  Requirements
  Install at least one of the following packages:

  • allure-python-commons
  • allure-pytest
  • allure-behave

  👉 Install with:

  pip install allure-python-commons

🛠️ Fixes

• Android Multi-Screen Support
  • Improved handling of multiple displays on Android devices

---

Full Changelog: v0.22.4...v0.22.5

v0.22.4

What's Changed

• Feat/Add annotation function by @mlikasam-askui in #188
• docs: add docstring to AndroidVisionAgent class by @mlikasam-askui in #192
• feat(chat): select model of run with request params by @adi-wan-askui in #193

🚀 Features

• Element Annotation:

  • annotate() method: Generate interactive HTML files that visualize detected UI elements on screenshots. The generated HTML allows users to:
    • View bounding boxes around all detected elements
    • Hover over elements to see their names and text values
    • Click on elements to copy their text values to the clipboard

  from askui import VisionAgent
  
  with VisionAgent() as agent:
      # Annotate current screen and save to default 'annotations' directory
      agent.annotate()
  
      # Or specify custom screenshot and output directory
      agent.annotate(screenshot="screenshot.png", annotation_dir="htmls")

  Also works with AndroidVisionAgent:

  from askui import AndroidVisionAgent
  
  with AndroidVisionAgent() as agent:
      agent.annotate()

  • locate_all_elements() method: Retrieve all detected elements programmatically as a list of DetectedElement objects:

  from askui import VisionAgent
  
  with VisionAgent() as agent:
      detected_elements = agent.locate_all_elements()
      print(f"Found {len(detected_elements)} elements: {detected_elements}")
  
      # Access element properties
      for element in detected_elements:
          print(f"Name: {element.name}, Text: {element.text}")
          print(f"Position: {element.center}, Size: {element.width}x{element.height}")

  • New Data Models:
    • DetectedElement: Represents a detected UI element with name, text, and bounding_box properties, plus convenience properties for center, width, and height
    • BoundingBox: Represents element coordinates with xmin, ymin, xmax, ymax, plus convenience properties for width, height, and center

• Chat API Model Selection: Chat runs can now specify which model to use via the model parameter in the run creation request, allowing dynamic model selection per run instead of using only the configured default model.

📜 Documentation

• AndroidVisionAgent class: Added comprehensive docstring with detailed parameter descriptions and usage examples for the AndroidVisionAgent class (src/askui/android_agent.py:41-62).

Full Changelog: v0.22.3...v0.22.4

v0.22.3

What's Changed

• Remove roman themed tone and voice by @onur-askui in #189
• Thinking is disabled and temperature is set to 0.0 for the Android Agent by @philipph-askui in #190

New Contributors

• @philipph-askui made their first contribution in #190
• @MichaelMayer-askui made their first contribution in #182

Full Changelog: v0.22.2...v0.22.3

v0.22.2

What's Changed

• Wait until by @programminx-askui in #178
• add askui controller client settings by @mlikasam-askui in #181
• fix anthropic integration + add push and pull to android agent os by @mlikasam-askui in #185

Full Changelog: v0.22.1...v0.22.2

v0.22.1

What's Changed

• fix: python 3.13.1 is now allowed again

Full Changelog: v0.22.0...v0.22.1

v0.22.0

What's Changed

• feat(agent): add offset and absolute position to click, type and move… by @programminx-askui in #174
• Fix python 3 14 by @programminx-askui in #177
• Feat/chat migrate persistence by @adi-wan-askui in #179

🚀 Features

• Offset and Absolute Positioning: Enhanced the click(), type(), and mouse_move() methods with more precise positioning capabilities:

  • Offset Parameter: Added offset parameter to fine-tune click and mouse positions relative to a target element

    # Click 10 pixels right and 5 pixels up from "Submit" button
    agent.click("Submit", offset=(10, -5))
    
    # Move mouse 5 pixels right and 10 pixels down from "Menu"
    agent.mouse_move("Menu", offset=(5, 10))
    
    # Type text in input field with offset
    agent.type("text", locator="Input field", offset=(5, 0))

  • Absolute Coordinates: Added support for Point tuples to specify exact screen coordinates

    # Click at absolute coordinates (100, 200)
    agent.click((100, 200))
    
    # Move cursor to absolute coordinates (300, 150)
    agent.mouse_move((300, 150))
    
    # Click at coordinates then type
    agent.type("username", locator=(200, 100))

  • Offset coordinates follow screen conventions: positive x moves right, negative x moves left, positive y moves down, negative y moves up

• SQLite Database Migration: Migrated chat persistence layer from JSON files to SQLite database for improved performance, reliability, and scalability:

  • Automatic Migrations: Database migrations run automatically on startup by default (configurable via ASKUI__CHAT_API__DB__AUTO_MIGRATE)
  • Migrated Components:
    • Assistants configuration
    • Messages, runs, and threads
    • File storage metadata
    • MCP (Model Context Protocol) configurations
  • Migration Framework: Added Alembic for database schema versioning and migrations
  • Backwards Compatibility: Original JSON files are preserved during migration, allowing easy rollback by installing an older version
  • Migration Documentation: Comprehensive documentation added in docs/migrations.md covering:
    • Migration strategy and execution flow
    • Manual migration commands
    • Troubleshooting and best practices
    • Database configuration options
  • Migration Commands:

    # Run all pending migrations
    pdm run alembic upgrade head
    
    # Show current migration status
    pdm run alembic current
    
    # Disable auto-migration for debugging
    export ASKUI__CHAT_API__DB__AUTO_MIGRATE=false

🐛 Bug Fixes

• Python 3.13 Support: Updated maximum Python version requirement to 3.13 as version 3.14 is not yet supported

Full Changelog: v0.21.1...v0.22.0