Skip to content

Commit 8cab9fd

Browse files
committed
Align Daytona computer-use shims
1 parent c9ee2fa commit 8cab9fd

7 files changed

Lines changed: 579 additions & 93 deletions

File tree

AGENTS.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -117,8 +117,9 @@ Useful flags:
117117
- `--sandbox-mode docker|local` selects static agent isolation. Use `local` only in a disposable
118118
outer sandbox such as Daytona, ephemeral CI, or a throwaway VM.
119119
- `--execute-app` enables opt-in live verification in a disposable runtime workspace.
120-
- `--computer-use daytona` exposes Daytona Toolbox-backed `cu-info`, `cu-screenshot`,
121-
`cu-click`, `cu-type`, `cu-key`, `cu-scroll`, and `cu-drag` commands to live Pi agents.
120+
- `--computer-use daytona` exposes Daytona Toolbox-backed `cu-*` commands to live Pi agents,
121+
including display/window inspection, screenshots, click aliases, mouse movement/button control,
122+
keyboard input/hotkeys, scrolling, dragging, and raw API calls.
122123
PITHOS calls the already-running sandbox Computer Use API on `127.0.0.1:2280`;
123124
do not require a client-provided Daytona Toolbox URL.
124125
- `--runtime-profile <path>` points at an explicit `.pithos/runtime.yaml` profile.
@@ -140,17 +141,21 @@ When Daytona computer use is requested, enforce all of the following behavior:
140141
- Require local runtime execution: `--sandbox-mode local` or `environment.sandbox: local` must be used
141142
so `127.0.0.1:2280` resolves to the outer Superagent sandbox daemon. Treat Docker runtime execution
142143
as a preflight issue for Daytona computer use, not as a soft warning.
143-
- Install `cu-info`, `cu-screenshot`, `cu-click`, `cu-type`, `cu-key`, `cu-scroll`, and `cu-drag`
144-
before starting the live verification agent, and prepend the shim directory to the agent `PATH`.
144+
- Strip Daytona and Superagent sandbox-creation credentials from the live-agent environment
145+
(`DAYTONA_*`, `PITHOS_DAYTONA_*`, and `SUPERAGENT*`). The agent should only receive local
146+
`127.0.0.1:2280` computer-use access, never credentials that can create more sandboxes.
147+
- Install the full `cu-*` shim set before starting the live verification agent, and prepend the shim
148+
directory to the agent `PATH`.
145149
- Run or require a smoke check (`cu-info` or direct `GET /computeruse/display/info`) before relying on
146150
Daytona Computer Use. If the daemon is unreachable, mark runtime verification blocked with a clear
147151
setup error instead of silently falling back to non-UI verification.
148152
- Route findings that need browser, desktop, terminal UI, OAuth/device flow, or visual interaction
149153
through the live-agent path when Daytona computer use is requested. Deterministic source-oracle
150154
plugins may still run for purely static/source-verifiable findings, but they must not be presented as
151155
evidence that Daytona computer use was exercised.
152-
- The live-agent prompt must require `cu-info` and `cu-screenshot` before UI actions, and another
153-
screenshot after UI actions. The verdict must cite the relevant computer-use evidence.
156+
- The live-agent prompt may allow normal `read`/`bash` investigation, but it must require `cu-info`
157+
and `cu-screenshot` before UI actions, another screenshot after UI actions, and computer-use
158+
evidence before accepting a verdict.
154159
- If a Daytona-enabled live-agent verdict does not include transcript or artifact evidence of at least
155160
one `cu-*` command, treat that verdict as blocked or inconclusive rather than confirmed or
156161
not-reproduced.

README.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -103,9 +103,10 @@ pithos run /path/to/repo \
103103
```
104104

105105
PITHOS does not start Daytona, Xvfb, XFCE, x11vnc, or noVNC. It creates
106-
per-run `cu-info`, `cu-screenshot`, `cu-click`, `cu-type`, `cu-key`,
107-
`cu-scroll`, and `cu-drag` shims that call the already-running sandbox Computer
108-
Use API on `127.0.0.1:2280`.
106+
per-run `cu-*` shims for display/window inspection, screenshots, mouse clicks
107+
and movement, keyboard input/hotkeys, scrolling, dragging, and raw Computer Use
108+
API calls. The shims call the already-running sandbox Computer Use API on
109+
`127.0.0.1:2280`.
109110

110111
If setup is incomplete, PITHOS writes `verify/RUNTIME-SETUP.md` with the missing
111112
env vars, personas, mocks, or profile fields. Live verification does not load

docs/configuration.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -159,9 +159,12 @@ pithos run /path/to/repo \
159159
--computer-use daytona
160160
```
161161

162-
PITHOS creates per-run `cu-info`, `cu-screenshot`, `cu-click`, `cu-type`,
163-
`cu-key`, `cu-scroll`, and `cu-drag` commands and prepends them to the live
164-
agent's `PATH`. The commands call the already-running sandbox Computer Use API
165-
on `127.0.0.1:2280`. The live-agent prompt tells Pi to screenshot before and
166-
after UI actions and to use `cu-info`, `DISPLAY`, and `xdpyinfo` diagnostics when
167-
desktop actions do not affect the expected screen.
162+
PITHOS creates per-run `cu-*` commands and prepends them to the live agent's
163+
`PATH`. These include display/window inspection, screenshots, click aliases,
164+
mouse movement/button control, keyboard input/hotkeys, scrolling, dragging, and
165+
`cu-api` for raw Computer Use API calls. The commands call the already-running
166+
sandbox Computer Use API on `127.0.0.1:2280`. The live-agent prompt tells Pi to
167+
use normal investigation tools as needed, but always verify through `cu-*`
168+
computer-use evidence before accepting a runtime verdict. PITHOS strips
169+
Daytona/Superagent sandbox-creation credentials from the live-agent environment
170+
so the agent can control only the existing local computer-use daemon.

0 commit comments

Comments
 (0)