Skip to content

Latest commit

 

History

History
214 lines (166 loc) · 11.1 KB

File metadata and controls

214 lines (166 loc) · 11.1 KB

Cua Driver

Background computer-use driver for any agents. Speaks MCP over stdio; drives native macOS apps without stealing focus.

See MCP protocol and skills for the modern stdio profile, legacy compatibility, bundled skill resources, and HTTP limits.

Documentation - Installation, guides, and API reference.

Integration surfaces

  • Cua as an agent MCP or CLI: MCP-capable agents connect directly to cua-driver mcp; shell-oriented agents and automation can use cua-driver call. No generated Cua language client is required.
  • Cua as an application SDK: Python applications import cua_driver; TypeScript applications import @trycua/cua-driver. Both package roots call the same in-process native runtime through generated UniFFI bindings. The safe Rust SDK and generated bindings sit above a versioned C ABI; they do not require a daemon for direct application use.

The language packages are for client applications, not agents. They contain no language-native MCP facade and have no /sdk, /mcp, or /native public suffix. MCP remains implemented by the cua-driver executable as the runtime-neutral agent boundary.

Computer History macOS preview

Nightly macOS builds can provide an opt-in, encrypted history of actions performed through Cua Driver. The preview stores a strict metadata allowlist, stays local, and exposes permission-gated history_status and history_query tools for read-only agent hydration. It never stores screenshots, typed text, clipboard contents, raw arguments or results, accessibility trees, paths, window titles, or URLs.

See Try the Computer History macOS preview for installation, lifecycle, inspection, deletion, and stable-channel return instructions. The architecture and staged plan defines the format, security boundary, release gates, and later NVIDIA OpenShell integration.

Permission modes

standard is the promptless default for normal automation. bounded admits only the tools and resources in a reviewed manifest. unrestricted requires --dangerously-bypass-approvals.

The mode belongs to the process that owns the runtime and is fixed at launch: cua-driver serve takes the flags, while cua-driver mcp and embedding hosts use the matching CUA_DRIVER_PERMISSION_MODE, CUA_DRIVER_CAPABILITY_MANIFEST_FILE, and CUA_DRIVER_CAPABILITY_MANIFEST_APPROVED variables. Choose it before starting the daemon; a running daemon must be restarted to change it.

Attaching to an existing logged-in Chromium profile remains explicit:

cua-driver mcp --grant existing-profile

An embedding application can instead provide DriverAuthorizationHost, and a bounded runtime can declare kind: existing_profile in its manifest. Cua Driver does not render its own authorization modal or banner.

See the hosted permission mode reference.

Repository Layout

Path Purpose
rust/ Cargo workspace for the daemon, UniFFI SDK, platform crates, and tests
python/ Python SDK, bundled-binary wrapper, and package tests
contract/ Experimental generated SDK contract and fixtures
typescript/ Generated TypeScript SDK
tests/fixtures/ Source-built GUI harness apps and shared fixtures
rust/crates/cua-driver/tests/ Rust integration tests for the driver and GUI harnesses
scripts/ Install, uninstall, local build, and VM sync helpers
docs/ Small repo-local specs that are not part of the hosted docs site

Start with rust/README.md, rust/crates/cua-driver/tests/README.md, and tests/fixtures/README.md when changing driver behavior or tests.

The contract-first SDK architecture is documented in contract/README.md. The Rust contract crate generates the checked-in manifest and supplies the typed inputs used by the live daemon and UniFFI SDK.

The imported SDK bindings are generated by scripts/generate-uniffi-bindings.mjs. CuaDriver.create() loads the native runtime in the application process; the daemon-compatible connect() path is retained for external clients. The stable native boundary is declared in rust/include/cua_driver_abi.h. That header is generated from the Rust #[repr(C)] exports with cargo run -p cua-driver-bindgen --bin cua-driver-abi-header; CI runs the same command with --check so the implementation and distributed header cannot drift. UniFFI separately generates its private FFI scaffolding and the Python and TypeScript SDK bindings; that implementation-specific scaffolding is not the stable public cua_driver_*_v1 C contract. Python wheels and the root/native npm packages are assembled from the same cua-driver-rs-v* release artifacts and publish with the exact Rust release version; .github/scripts/validate_release_versions.py rejects source drift.

For direct agent integrations, see the examples/agent-sdks Codex and Claude Agent SDK examples. They connect the agent to cua-driver mcp without importing a generated Cua client.

Muse Code

Muse Code can use Cua Driver as a local stdio MCP server on macOS and Linux. Merge the following entry into $XDG_CONFIG_HOME/muse/settings.json, or ~/.config/muse/settings.json when XDG_CONFIG_HOME is not set:

{
  "mcp_servers": {
    "cua-driver": {
      "enabled": true,
      "transport": "stdio",
      "command": "/absolute/path/to/cua-driver",
      "args": ["mcp"]
    }
  }
}

Use the installed binary's absolute path, which command -v cua-driver prints. Merge the cua-driver entry with any existing mcp_servers instead of replacing the settings file. Start a new Muse session after saving the file; Muse loads MCP servers at session startup.

Contributor documentation:

  • docs/cursor-themes.md documents the default semantic cursor and custom dotLottie authoring contract.
  • docs/test-matrix.md maps unit and canonical harness E2E suites.
  • docs/action-support.md is the empirical platform behavior ledger.
  • docs/test-harnesses-guide.md explains fixture and runner ownership.
  • docs/linux-desktop-validation.md covers representative Linux sessions.
  • docs/linux-support-completion-plan.md preserves the historical Linux plan.

Claude Code computer-use compatibility

Standard Claude Code MCP registration:

claude mcp add --transport stdio cua-driver -- cua-driver mcp

If you want Claude Code's vision/computer-use-style flow to ground on CuaDriver window screenshots, register the compatibility mode:

claude mcp add --transport stdio cua-computer-use -- cua-driver mcp --claude-code-computer-use-compat

This keeps CuaDriver's normal MCP tools and changes only screenshot, which requires pid and window_id and captures that window only.

Use MCP for this Claude Code vision/computer-use-style path. CLI screenshots still work as CuaDriver calls, but they do not expose the mcp__cua-computer-use__screenshot tool name that Claude Code appears to use as the image-grounding cue.

macOS process identity and permissions

macOS attributes Accessibility and Screen Recording grants to a responsible app identity, not simply to an executable path. Use one of these supported launch modes:

  • Standalone: install CuaDriver.app, grant permissions to it, and start its daemon with open -n -g -a CuaDriver --args serve. The installed cua-driver mcp CLI may proxy through this daemon automatically.
  • Explicit direct MCP: cua-driver mcp --direct makes the MCP process own its runtime. On macOS this deliberately uses the spawning host's TCC attribution and does not provide the AppKit cursor overlay without a certified host adapter.
  • Embedded: have the macOS app that owns the grants use the generated EmbeddedCuaDriverHost to spawn a private daemon and return both SDK and MCP connection details. The daemon stays in the app's responsibility chain and inherits its grants. A gateway, terminal, or unrelated helper must not spawn the daemon on the app's behalf. @trycua/cua-driver/embedded is an organizational alias for the same Rust host exported at the package root; it has no separate lifecycle implementation.

Directly spawning a raw cua-driver serve outside CuaDriver.app without embedded mode is unsupported: it has no stable bundle identity for TCC attribution. Do not grant permissions to arbitrary binary paths or rely on that configuration in production. See rust/Skills/cua-driver/EMBEDDING.md for the embedding contract and examples.

Publishing the agent skill to ClawHub

The canonical skill source is rust/Skills/cua-driver. It is published as one cross-platform ClawHub skill at @cua/driver; the bundle includes the macOS, Windows, and Linux documents. Direct installs through cua-driver skills install still keep only the host OS document unless --all-platforms is used.

ClawHub releases have their own explicit license boundary. The repository stays under MIT, while every skill copy published through ClawHub is distributed under MIT-0. Before a release, retain an internal record that Cua AI has the right to distribute every bundled file under MIT-0.

Pull requests that change the skill run a publish dry-run with the pinned ClawHub CLI. A real release is available only through the ClawHub: cua-driver skill workflow's manual dispatch. The publish job requires all of the following:

  1. Dispatch the workflow from main.
  2. Enter a version that matches both rust/Cargo.toml and the version field in rust/Skills/cua-driver/SKILL.md.
  3. Confirm the MIT-0 rights check in the workflow form.
  4. Configure a repository Actions secret named CLAWHUB_TOKEN for a publisher that can release under the selected owner. The default owner is cua.

The workflow pins the ClawHub CLI, records the source repository, commit, ref, and path, and uploads the JSON publish result as an Actions artifact.

After publishing, inspect and scan the exact version, then install it into an empty work directory:

npx --yes clawhub@0.23.1 inspect @cua/driver --version 0.8.3 --files
npx --yes clawhub@0.23.1 scan --slug driver --version 0.8.3 --update
npx --yes clawhub@0.23.1 --workdir /tmp/cua-driver-clawhub-smoke \
  install @cua/driver

Confirm that MACOS.md, WINDOWS.md, and LINUX.md are present, run cua-driver doctor, and perform a read-only list_apps call through OpenClaw. If a release is faulty, publish the last known-good content as a new patch version. Do not delete the current latest version before a replacement exists.