Date: 2026-06-27 · Scope: what is actually built and proven vs designed-only vs unvalidated. The design corpus (vision/PRD/architecture/ADRs/roadmap) intentionally describes the full CUA infrastructure stack. The built implementation spans a cross-platform D15 backend waist and Shinken-owned native runtimes; the deepest well-tested native slice is local Linux/X11. This page is the honest map between target scope and current code. When in doubt, this file describes what exists today; the roadmap describes what v0.0.1 and later releases must add.
Audience: users and implementers · Role: implementation reality check. This is the source of truth for what exists today.
The one-line summary: Shinken currently ships a cross-platform, backend-pluggable CUA environment/runtime and rollout data plane behind one typed ACI. Built operation-layer backends reach Linux, macOS, Windows, and browser surfaces; proof depth varies by backend. Shinken's deepest first-party native-runtime implementation and automated validation are currently on Linux/X11, with a native macOS v1 supported by local-only proof. It does not yet ship a distributed fleet control plane, durable trajectory/artifact service, token-exact policy/trainer integration, CoW/microVM fast tier, or native Windows/Wayland/full-macOS runtime.
The Proof column states the actual evidence level for each capability and platform. Linux/X11 has the deepest native live-CI coverage; macOS native v1 has local-only proof; the cross-platform backends have fixture coverage plus backend-specific live gates. Exceptions such as external drivers not yet live-run and the post-hardening CRIU path are called out explicitly.
| Capability | State | Proof |
|---|---|---|
| ACI v0 handshake + capability negotiation | done | shinkend + SDK; unit + live |
| Secure transport (handshake-first state machine, token required on every TCP listener, browser-Origin deny-by-default) | done | #38 + 2026-06 P0 hardening |
| Honest capability negotiation + ACI version/unknown-field enforcement | done | #39 |
| Pointer actions (move/click/double/right/scroll) via X11 XTEST | done | live Xvfb smoke (cursor verified with xdotool) |
Gesture verbs: drag (down → interpolated path → up; duration_ms/button) + decomposed mouse_down/mouse_up |
done | Rust unit (path/button/dispatch) + SDK mock + contract gate + live Xvfb smoke (scripts/action_surface_smoke.py: drop point verified with xdotool) |
Keyboard (type_text, key, keysym + modifier combos incl. xdotool-style chords, numpad kp_*, named symbol keys) |
done | #30 + keysym unit tests |
Act-returns-observation (observe on every mutating verb: ack + fresh observation with cause=call_id, same round trip; capabilities.observe_after_act-gated, old runtimes/clients unaffected) |
done | Rust unit (validation-before-execute, binary/text follow-up, honest-ack-on-capture-failure) + SDK demux tests + contract gate + live Xvfb smoke |
list_windows query (EWMH _NET_CLIENT_LIST/_NET_WM_NAME/_NET_WM_PID/_NET_ACTIVE_WINDOW; WM-less query-tree fallback) → [{id, title, pid, x, y, w, h, focused}] |
done | Rust unit + SDK mock + live Xvfb smoke (enumerates the CI xclock) |
Typed in-guest exec channel (exec verb, G1): argv default + explicit shell opt-in, cwd/env/timeout_ms/stdin; buffered typed result ($defs.ExecResult — exit/signal/timed_out, capped stdout/stderr with honest truncation flags) + streamed form (exec_output events on the demux, raw-byte binary frames on binary sessions, terminated by one exec_exit); process-group kill on timeout, zombie reaping (kill_on_drop), bounded concurrency (semaphore, saturation nacks); gateway exec capability with the argv/shell recorded in the audit event's detail; swerex/CUA-Gym/ProRL integrations prefer it over docker exec when advertised (out-of-band fallback kept) — exec now works on substrates where docker exec doesn't exist; pty reserved (false-only; PTY = designed follow-up) |
done | Rust unit (validation, timeout group-kill, output caps, stream ordering/budget) + Rust WS e2e + SDK mock tests (tests/test_exec.py) + contract gate + integration preference tests + Docker-gated live test + live smoke vs the real runtime (echo/env/cwd/exit/truncation/timeout/stream/audit) |
Desktop verbs (G2+G3): clipboard_get/clipboard_set (Linux: shinkend speaks the ICCCM CLIPBOARD selection protocol itself on a dedicated worker thread — owner serves TARGETS/UTF8_STRING/STRING, get via ConvertSelection + bounded wait + one STRING retry; no xclip; v1 text-only, 1 MiB cap, INCR refused typed; the read answers a typed result {text}), launch_app {app, args?} (detached spawn with the session $DISPLAY + D-Bus env, no shell, background reaper; find the window via list_windows), activate_window {window_id | app} (EWMH _NET_ACTIVE_WINDOW client message; WM-less raise+set-input-focus fallback, with active_window gaining the matching input-focus fallback). The writes are MutatingVerbs (observe-after-act works); SDK gateway gates clipboard both ways on clipboard (default-off), launch on app_launch. macOS v1: typed unsupported errors (needs AppKit bindings) |
done (Linux v1) | Rust unit (selection state machine, EWMH message shape, spawn/reaper, schema-mirror contract tests) + SDK mock + contract gate + live smoke scripts/clipboard_app_smoke.py in BOTH desktop shapes (Docker/openbox EWMH path AND bare-Xvfb fallback path: set→get roundtrip ×2, launch xlogo → enumerate → activate → active-window confirmed) |
| Screenshot capture (X11 GetImage → PNG) | done | live Xvfb + Docker smoke |
| Real-time screencast (server-pushed frames, single-writer transport) | done | #48 — tokio WS integration test + live |
Bandwidth levers: idle-frame suppression + resolution downscale (max_long_edge) |
done | #48/#54 — live: 1280×800→640×400, ~3.4× smaller |
JPEG observation codec (format/quality per action, capability-negotiated) |
done | #243 — measured ~1–21× vs PNG, content-dependent (benchmarks §2) |
| Lossless dirty-tile delta screencast (changed 64-px tiles + periodic keyframes) | done | B2 — measured ~11× vs full-PNG while typing (benchmarks §3) |
SharedLoop (N sync sessions on one event-loop thread) + ping_jitter fleet decorrelation |
done | #244 — measured: 64 real sandboxes / 3,096 live ACI sessions (synthetic peers) on one loop thread, sustained 2,320 frames/s ≈ 870 Mbps at 0.93 cores (benchmarks §5–§6) |
| Rerunnable local benchmark suites (14 suites + pool/memory fork modes: codec/delta/latency/fork/fan-out/client-scale/wire-ceiling/OSWorld-head-to-head/guest-CPU/boot-waterfall/fork-dedup/step-pipeline/cua-baseline/legibility-envelope; raw JSON + figures tracked) | done | benchmarks/ → benchmarks.md |
Binary WS media frames (negotiated hello.accept.binary_frames ↔ capabilities.binary_frames; u32 LE header_len | JSON header | raw codec payload; text base64-JSON kept for old clients/runtimes) |
done | Rust + SDK + contract gate ($defs.BinaryFrameHeader); measured: wire −25%, client-plane ceiling ~1.9× (benchmarks §3/§4/§6b) |
XDamage event-driven capture (root-window DAMAGE on a dedicated X connection; an idle tick captures NOTHING, a damaged tick GetImages only the damage-bounding region and composes onto the delta baseline; periodic keyframe kept as the correctness backstop; SHINKEND_DAMAGE=off reverts to poll-diff) |
done | Rust unit + e2e idle-zero tests, live Xvfb probe (idle silence + damage wake), delta reconstruction pixel-identical in-suite; guest CPU measured in benchmarks §8 |
Focused-window / region capture (scope: screen / active_window / window:<id>) |
done | #55 — live xclock window:<id> → 200×200 |
Guest structured-observation engine v1 (Linux/AT-SPI) — observe verb in shinkend: AT-SPI walk of the focused app on a dedicated hang-isolated worker (per-call/walk/node caps; a hung bus answers a typed error, never wedges the runtime), stable element ids (e<N>, monotonic, composite path+role+parent identity, evicted-never-reused, machine-readable stale_element_ref: on stale refs), legible tree_text (numbered indented lines + header + focus trailer) with ~/+/- diff rendering vs the previous revision (line-budgeted, removed-id ranges), settle (bounded a11y-event debounce), element_ref action targets resolved guest-side (bbox-centre XTEST click), and the invoke_action/set_value AX-path verbs; advertised as capabilities.structured_observation; SDK observe(structured=True)/observe_diff() prefer the guest engine with the SDK-local AT-SPI helper kept as fallback |
done (Linux v1) | Rust unit (fake tree source) + WS e2e + SDK mock tests + contract cases + live Docker smoke (scripts/observe_smoke.py: zenity --entry — id stability, diff after typing, element click by id, invoke_action) |
| Python SDK: sync facade + async core, reader/demux (RPC vs server-push) | done | #51 |
Pipelined step() — k actions + a fused trailing observation in ~1 RTT (no protocol change; honest no-skipped failure rows, #56 taxonomy) |
done | unit-tested vs mock (single-flight, partial failure, observation fusion, death mid-step) + S11 WAN-emulation bench (benchmarks/bench_step_pipeline.py) — benchmarks.md §11, many-sandbox-concurrency.md §5 |
TypeScript control-surface SDK (sdk/typescript/) |
done | tracked + CI-tested (dedicated SDK (TypeScript) job) |
OSWorld DesktopEnv compatibility shim |
done | #32 |
| CU provider adapters (Anthropic, OpenAI, Kimi-VL) → canonical ACI | done | fixture-tested, no live API; #75/#76 + Kimi-VL |
String-form XML tool calls as first-class action input (shinken.dialect.parse_xml_actions + parse_actions(text, format="auto"|"xml"|"dialect"): JSON-in-XML <tool_call>, <invoke>/<parameter> blocks, <function=…> element XML, attribute/element XML; tolerant recovery, typed DialectError on unknown/unsupported — never a silent drop; adapters expose .from_text) |
done | corpus-driven tests/test_dialect_xml.py (39 real-shaped samples) · aci-spec §3.1 |
RL rollout-server runtime plugin (ProRL-Agent-Server BaseRuntime contract — integrations/prorl_agent_server.py, INIT = resume-from-golden) |
done | fixture-tested, no hard dep; live Docker roundtrip env-gated (SHINKEN_DOCKER_LIVE=1) · agent-runtime.md |
NeMo Gym computer-use resources engine (integrations/nemo_gym.py): fork-from-golden seed, generation-fenced tool/verify/end/reseed, same-session linearization, active abandoned-rollout reaping, leased TTL/LRU-bounded golden ownership, terminal shutdown barrier; backend/OS-neutral lifecycle with Linux/X11 demo evidence |
done (single-process instance) | deterministic fixture tests cover concurrent seed/tool/verify/close/reap and deletion retry; real two-demo Docker loop + recorded NeMo Gym rollout collection · example. Multi-worker startup fails closed; horizontal session routing/control plane is not built |
Agent-runtime narrow waist (shinken.runtime: Session/rollout/Trajectory, zero scorer/reward) + Workload registry |
done | #220/#221/#227 · agent-runtime.md |
Provider registry + DockerLocalProvider + out-of-tree plugin loaders |
done | #219/#226 |
Pluggable shinkend injector (shinken.inject: docker/ssh/osworld-exec) |
done | #230/#233 — chunked upload, shell-wrapped start, configurable remote path, surfaced errors |
OSWorld as a Workload (osworld-eval) + scripts/osworld_single.py runner |
done | #222/#229 — Kimi K2.6 in OSWorld pixel-pyautogui form, scored by the official OSWorld evaluator |
Tiny eval harness + run_eval_forked (golden-checkpoint → fork-N → score) |
done | #86/#87/#206/#231 — non-vacuous verifiers, unit-tested |
Push-based boot readiness (S9): guest-side ready query ({ready, x11_up, root_nonblack} from sampled 1×1 root pixels), lazy/self-healing X11 connect in shinkend, shinkend exec'd before the desktop, SDK readiness = ONE persistent WS + 15 ms guest polls |
done | measured: provider.create() ~7.7 s → ~0.19 s p50; full waterfall in benchmarks §9 |
| Warm-pool filesystem graft | disabled | The historical live-target graft raced the target's own writers and could diverge from cold restore. warm_pool_size>0 now fails explicitly until an equivalence-preserving stopped-target design and pool-hit/pool-miss parity gate exist; historical S4b data remains in benchmarks §1, not as a current claim |
CRIU memory tier (opt-in CriuDockerProvider, snapshot_kind="process", privileged-only, Linux/Docker-only): checkpoint uses criu dump --leave-stopped, commits filesystem state in the same stopped window, then resumes the donor; restore = fresh privileged container + parked-PID criu restore. Explicit state_fidelity="process_memory" routing prevents disk-tier substitution |
hardened; live rerun pending | Historical S4c smoke proved an in-heap marker and donor liveness, but its 0.70 s / 0.40 s timings used the prior split-time path. The atomic implementation is unit-tested; privileged Docker fidelity + latency rerun is required before republishing those numbers |
uni-agent/verl interop: swerex-shaped deployment/runtime (shinken.integrations.swerex, fork-from-checkpoint start()) |
done | protocol-shape unit tests + live Docker e2e (agent-runtime.md) |
Checkpoint-native gym adapter (shinken.gym): trainer-facing make/reset/step/evaluate; reset() restores a new replica; reset-strategy resolution (auto/fork/recreate): fork-from-golden where the provider serves the checkpoint+resume pair, honest recreate fallback (fresh sandbox + per-episode task.setup replay, loud degrade) over snapshot-less providers — the gym harness runs over the D15 backends, with info["reset_strategy"]/trajectory metadata recording which path ran; recreate requires a real create/destroy lifecycle (attach-only ExternalProvider stays a typed failure) and run_eval_forked still requires a snapshot tier; transitions record (s_t,a_t,s_{t+1}) with a semantics marker and the HF exporter pairs the policy input with its action. ShinkenGymPool(n) shares one SharedLoop/FrameCache and the strategy resolution; provider fidelity is preflighted |
done (Docker live rerun pending for provider hardening) | trajectory/exporter/dataloader fixture tests incl. the recreate-fallback set (auto probe/degrade, capability skip, strict fork, pool fan-out, setup-failure teardown); the old warm-pool latency gate is historical and disabled; example examples/gym_rollout.py · agent-runtime.md |
Content-negotiated observation / fleet-level frame dedup (screenshot.if_none_match ↔ runtime frame_hash over RAW pixels — codec-independent XXH3-128, hardened from fnv1a-64 (collision = silently wrong pixels; analysis in aci-spec §4) — + compact not_modified; capability-negotiated frame_dedup; SDK per-session dedup + shared shinken.FrameCache across a forked fleet's sessions) |
done | S10 — Rust unit + SDK mock + contract gate both sides, stale-old-format-hash mix test; measured over forked fleets N ∈ {4,8,16}: sequential static ceiling 18.6× wire cut / 94.6% hit rate / ~654× steady state; concurrent (asyncio.gather, trainer shape) round-0 first-touch race = 0/28 hits then 1.0 steady; policy-driven full divergence decays to 0.0 hit rate (≈1× bytes — the honest floor); replicas verified marker/pixels/fs 28/28 each (benchmarks §10) |
Agent-quality study harness (codec tier × task success, fork-eliminated env variance): 16 read-from-screen-critical tasks (8 templates × normal/small font, deterministic exec-state verifiers), one golden checkpoint per task with every tier×seed episode forked from it, pluggable OpenAI-compatible vision driver (SHINKEN_STUDY_* env, temp 0, parse_actions auto) + scripted-oracle plumbing mode, Wilson-CI / −5 pp non-inferiority analysis with the Arm-1 legibility join point on tier labels |
harness built; pilot run, full study pending | oracle-validated over the whole corpus (benchmarks/results/agent_quality_oracle_plumbing.json) + real-model pilot (benchmarks/results/agent_quality_pilot.json) · agent-quality-study.md |
Operation-layer backends (D15, shinken.backends): third-party computer-control drivers plug in under the typed ACI as a SandboxProvider returning a duck-typed Sandbox — honest capability negotiation (advertise only the verbs/targets/observation tiers actually served; missing capability ⇒ typed UnsupportedProviderOperation, never a silent no-op or faked tree; no fork tier ⇒ supports_fork=False). Built adapters: cua (trycua BaseComputerInterface), mcp-computer (codex-style AX MCP — serves the structured element_ref family, fills the macOS-AX gap), browser-runtime (the D13 §10 Browser Runtime as a CDP backend, three tab surfaces), e2b (E2B cloud desktop, pixels + shell). RoutedSession composes CU↔BU surfaces behind one Sandbox-shaped object the Operator loop drives unchanged, routing per action with source provenance |
done | fixture-tested with protocol-faithful in-memory peers (no SDK/VM/key) — tests/test_backends_{cua,mcp_computer,browser_runtime,e2b,routed}.py; env-gated live smokes against the real drivers — tests/test_backends_live.py (SHINKEN_{BROWSER,E2B,CUA,MCP}_LIVE=1; browser proven locally against real headless Chrome: real AX tree → element_ref click landed, typing observed via the page's own JS; e2b/cua/mcp gates written but unrun — each needs its external system; none in CI); examples examples/backends_*.py · tech-decisions D15 · operation-layer §13 |
| Wheel packaging (schemas bundled), idempotent close | done | #40 |
| Docker Linux sandbox image | done | #45 — build + token handshake + screenshot in CI |
CI: 9 jobs (guard, schema sanity, shinkend Rust, v0.0.1 contract gate, Python SDK, TypeScript SDK, wheel install, live Xvfb integration, Docker sandbox image) |
done | every PR |
| Item | Reality |
|---|---|
element_ref action targets |
Built for Linux v1: the guest observation engine resolves a live element_ref to its bbox centre and clicks there via XTEST (physical-event preference); invoke_action is the AX-path fallback for geometry-less elements. Stale/unknown refs answer a machine-readable stale_element_ref: error (= re-observe). Honest limits: identity rides on AT-SPI object paths (+role+parent) — a toolkit that recycles a path for an identical-role sibling under the same parent is indistinguishable from the original; coverage follows the spike's measurements (Qt strong, GTK weak, terminals absent, canvas zero — E5), so the hybrid default stands and pixels remain the baseline |
| Wire schema vs implementation | The screencast wire vocabulary (start_screencast/stop_screencast, screencast, scope, fps, max_long_edge, stream/seq, resume_stream) is validated by schema/aci.schema.json and exercised by the contract gate (#187); the authenticated handshake (hello.token) is schema-valid and contract-tested. The error taxonomy is now implemented (SandboxDied with exit/signal detail, typed per-action act_batch status + failure_kind, eval RunResult.kind/infra_failure, provider.check_alive()), and so is screencast reconnect (resume_stream on start_screencast: a live logical stream keeps its stream id with seq continuing — the frame gap readable off the first frame — while an expired one restarts fresh at seq 0). The trajectory-level exit_reason is implemented too: documented precedence sandbox_died > setup_error > agent_error > scorer_error > max_steps > task_complete (shinken/runtime/trajectory.py), set by rollout, the OSWorld episode/receipt, and eval RunResult.exit_reason (the finer projection of kind) — closing the #56 contract residue (v0.0.1-plan §6). The query vocabulary (Query.q) now includes ready (S9 guest-side readiness) — schema-enumerated, contract-gated on both sides (Rust query_vocabulary_matches_schema, Python wire-vocab cases), and back-compatible: an old runtime answers unknown query: ready and the SDK falls back to a same-connection screenshot probe |
| Hardening | The 2026-06 review sweep landed the queue bounds (writer channel bounded + default frame-size cap), the pre-auth upgrade/write deadlines, the vacuous-test fixes, and the typed failure taxonomy (recalibration inventory §3–§5, C‑4/T‑5); the screencast reconnect contract, the trajectory-level exit reason, and subprocess scorer isolation (T‑5: shinken/scorer_proc.py — fresh subprocess, atomic result file authoritative over exit code/timeout, typed ScorerError → exit_reason="scorer_error", default-on for the external osworld-eval evaluator) have since landed too, closing the #56 contract residue |
| Runtime state (snapshot/restore/resume/fork/checkpoint) | Implemented on the Docker filesystem tier with immutable Docker image IDs behind opaque checkpoint IDs, persisted image-label metadata, and typed SandboxSpec preflight (os, GUI, GPU, fast reset, state fidelity). Human names are labels only. Warm graft is disabled; CRIU uses one stopped consistency window and requires privileged live revalidation. Still missing: the sub-ms CoW fast tier (#206). |
| Local capability gateway (Action Gateway shim + capability envelope + ask-tier) | Built as an SDK-process shim (sdk/python/src/shinken/gateway.py + tests): records the session capability envelope (the declared resource scopes) and routes boundary requests through an allow/ask/deny decision. This is the v0.0.1 audit/policy seam for eval and runtime resource-scoping, not the production control-plane resource-scoping layer (no Cedar/ocap/OS layer) — see Designed-only below |
macOS engine v1 (native shinkend backend) |
Capture + input built, CI local-only. shinkend/src/executor_macos.rs (cfg + macos-native feature; Linux build untouched, cross-checked vs aarch64-unknown-linux-musl): CoreGraphics capture (full-screen + window:<id> + frontmost-window active_window; deprecated-but-functional CG APIs, ScreenCaptureKit upgrade path documented) feeding the SHARED encode pipeline; CGEvent input (move/click/double/right/scroll/drag (schema shape: target→to)/mouse_down/mouse_up/type_text/key — US-ANSI combo keymap, layout-independent type_text, super+… = ⌘); Retina pixel↔point scaling (ACI space = capture pixels); TCC-honest readiness (display_up + permissions_pending on the ready query, typed permission_pending: refusals instead of wallpaper frames/dropped events); --backend macos flag + auto-detect when $DISPLAY absent. Proof: 15 cfg-gated unit tests + live local smoke (scripts/macos_smoke.py) on a real Mac — pointer move verified at exact Retina scale; full-content capture pending the host's Screen Recording grant (the typed-pending path is live-verified). No macOS CI runner yet; AX-tree observation designed-only. See macos-engine.md |
The remaining pieces named below are not implemented, even where an adjacent capability or backend path is already built:
- First-party native runtime coverage: the cross-platform backend layer is built (D15, see the implemented table above); what is still designed-only is Shinken's native Windows engine and Wayland engine. macOS has a native v1 engine — capture + input built with local-only proof (see Partial above; no mac CI, AX observation not built). The current native Linux capture/input path is X11 only; Wayland would require a different implementation than XTEST/GetImage.
- Structured / accessibility observation (ADR D3) — the Linux/AT-SPI guest engine v1 is now BUILT (see Implemented above:
observe+ stable ids + diff + settle + guest-side element_ref +invoke_action/set_value, live-smoked against zenity). Still designed-only within D3: the in-guest CDP backend (browser/Electron trees — todaycdp.pyremains SDK-local), Set-of-Marks, Windows UIA / macOS AX, and the structured-default posture itself — the spike's verdict (hybrid: weak GTK coverage, canvas measured at zero, E5) stands, so structure is an upgrade path over the screenshot baseline, not the default. - Operation layer (D13, operation-layer.md) — the core contract is now BUILT for Linux v1: the stable-id observation engine (
observewith~/+/-diff rendering, line budget + full-tree fallback, typed stale-ref + re-observe hint, settle-before-observe), act-returns-observation (per-actionobserveon every mutating verb), the coordinate-tier completions (drag,mouse_down/mouse_up), the element verbs (invoke_action,set_value— the canon'sinvoke_element_action/set_element_value, shipped under the shorter wire names), thelist_windowsquery, and the G2+G3 desktop verbs (clipboard_get/clipboard_set,launch_app,activate_window— see Implemented above). Still designed-only within D13:set_text_selection,scroll_element, the per-app observe selector (observe {app}), theappsquery, and hint packs. - macOS engine (D14) — the capture+input v1 slice is now BUILT (
shinkend --backend macos: CoreGraphics one-shot/raw capture + CGEvent pointer/keyboard/scroll/drag under the Executor trait, TCC-gated readiness; local-only proof viascripts/macos_smoke.py— no mac CI). v1 is exclusive-desktop semantics: input posts to the global HID tap and moves the real cursor. Still designed-only within D14: the co-use tier (per-app background input viaCGEventPostToPid+ a software cursor overlay for the human; until it lands, co-use is served by themcp-computerbackend, D15), the AXUIElement observation tier (incl. the Chromium/Electron accessibility-enable attributes, permissions-pending observe = typed keep-alive), and the ScreenCaptureKit upgrade path (macos-engine.md; readiness analysis: spike-macos-readiness.md). - Browser Runtime — the browser-specialized runtime profile (Playwright-locator-subset scripts /
semantic node-ids over the DOM / pixels via CDP) outlined in
operation-layer.md §10 is designed-only; only its CDP
coverage basis is first-party-measured (spike #2/E5). (Distinct from the built SDK-local
browser-runtimebackend (D15, see ✅ above), which drives an external CDP browser through the same three-surface shape — it is not this in-guest, Shinken-managed runtime.) - Capability Manager panel + production resource-scoping layer — the designed control-plane 3-layer model (Cedar decision + object-capability caretaker + OS-level scoping + egress proxy + secret broker) that scopes which resources a session can reach. Only the local Action Gateway shim exists today (see Partial, above); the production control-plane layer is not implemented.
- Runtime-state fast tier — the Docker disk tier and the CRIU memory tier (
snapshot_kind="process",CriuDockerProvider) are both implemented (see Partial + the ✅ table; the spike that gated the memory tier isspikes/criu-memory-tier/). What remains designed-only here: the sub-second CoW fork-from-snapshot fast tier (Firecracker/QEMU, Phase-1) and any incremental/lazy CRIU path (no SOFT_DIRTY/USERFAULTFD on the dev-machine kernel). The memory tier's standing caveats: privileged containers + root desktop tree (a latency/state-fidelity feature, never an isolation posture — the microVM tier remains the production boundary, D1), andat-spi-bus-launcheris replaced by a launcher-less single-bus a11y stack because CRIU 3.17 cannot dump pidfds (images/linux/start-criu.sh). Runtime-state time-travel is the headline differentiator (D1/D5, #206). - Backend runtime-state mapping (D15 extension) — mapping a third-party substrate's own snapshot/fork primitive into the backend provider's
checkpoint/resumesosupports_forkflips honestly toTrueand gym fork-reset rides it (today every D15 backend issupports_fork=False; the gym's recreate fallback is the built bridge). Candidate substrates with a real 1:N instantiate-from-snapshot API (all public docs, latencies vendor-published, unverified): cua Cloud (sb.snapshot()→ Image → N sandboxes; containers are filesystem-tier), Daytona (_experimental_fork/_experimental_create_snapshot(include_memory=…)— explicitly experimental-prefixed), Morph Cloud (instance.branch(count=N), memory+disk). E2B pause/resume is 1:1 persistence, not fork — excluded. Requirement:snapshot_kind/state_fidelitymust carry through capability negotiation so the gym's fidelity preflight keeps rejecting disk-tier substitution forprocess_memorytasks. See operation-layer.md §13. - Replay /
.sknrecording and playback — deferred to later design work; no runtime or SDK implementation is shipped now. - Control plane + ultra-high-concurrency / multi-tenant orchestration — a single local
shinkendis all that exists. - Dual-channel WebRTC media plane + GPU/NVENC encode — today streaming is PNG/JPEG frames over the control WebSocket (raw-byte binary frames on negotiated sessions; base64-JSON for old clients), fine for an MVP but not the low-latency/bandwidth story at scale.
- High-throughput file-transfer path (#49/#50) — design only.
- Code-agent RL readiness — the typed
execverb is now BUILT (see the table row: argv/shell forms, buffered + streamed, group-kill timeouts, gateway-audited; the swerex shim and the CUA-Gym/ProRL adapters prefer it in-band). Still reserved/design-only from this family: PTY allocation (the wire fieldptyis pinned,false-only), the headless (needs_gui=False) code-image profile, and the token-fidelity trajectory fields (code-agent-rl.md);shinken.inject+put_file/get_fileremain the substrate-side channels andrun_eval_forkedthe fork-N primitive. - Full OSWorld-Verified conformance at scale — the building blocks all ship (see the ✅ table: OSWorld-as-a-Workload, the
scripts/osworld_single.pyrunner driving Kimi K2.6 in OSWorld's native pixel-pyautogui code-block form —parse_model_actionsmirrors OSWorld'sparse_code_from_string— scored by the official OSWorld evaluator, theshinkendinjector for--backend shinkenactuation, andrun_eval_forkedfor the golden→fork-N→score loop). The open-weight Kimi-VL/Aguvis path is a separate adapter (adapters/kimi.py). What is not done here: a full OSWorld-Verified conformance sweep (Small/full set) with published numbers, and large-N forked scoring at scale.
The roadmap names these as the de-risking spikes:
- Spike A — a11y coverage (#2): MEASURED (E5), verdict in. Coverage of real app surfaces:
strong for Qt via AT-SPI (0.87 addressable) and Chromium-family-via-CDP (every labeled control
resolved; 0.23 of all nodes — measured on both the browser and a real Electron 35 app, which
also reaches AT-SPI at 0.32 but only under
--force-renderer-accessibility), weak for GTK (0.09–0.10), absent for terminals, and measured-zero for canvas-rendered UI (a page drawing 5 working controls inside one<canvas>exposes 2 inert AX nodes, and a real click repaints the screen while the tree diff reports 0 changes — the blind spot is quantified, not assumed); tree-diff ~2 KiB vs ~77 KiB screenshot while typing. Verdict: supports a hybrid per-window structured + pixel fallback, not structured-by-default — D3 stays Provisional (evidence:spikes/a11y-coverage/, summary in docs/benchmarks/). Games/native-GL surfaces remain unmeasured (the canvas row is the measured proxy). - Disk-tier fork economics: MEASURED — checkpoint ~0.6 s live, fan-out wall-clock flat in N (benchmarks §1). CoW-fork density (the designed sub-second fast tier, Phase-1 boundary) remains unmeasured.
- Dual-channel WebRTC latency (Phase-1 boundary) — latency target unmeasured.
- NVENC density / GPU-TEE attestation (Phase-4) — unmeasured.
- "M0/M1" in commit/PR labels delivered the Linux/X11 pixel + streaming slice above — real and tested. That is genuine progress and should not be undersold.
- The milestone is now v0.0.1 — feature-complete local/reference runtime: all core semantics should exist and be tested locally, even if performance, fork density, WebRTC/SFU/NVENC, multi-tenant control-plane operation, and cross-substrate scale come later.
- Milestone labels previously outran both the contract (schema drift, #56 — since closed) and the structured-observation evidence (Spike A — since measured, verdict: hybrid). Treat the 🔵/🔬 sections as the honest v0.0.1 and post-v0.0.1 backlog, not as a reduced product scope.
See also: roadmap, tech decisions D1–D15, hardening backlog (#56), a11y-coverage gate (#2).