| name | sandbox-verifiable-planning |
|---|---|
| description | Use when writing an implementation plan or executing one where some tasks cannot be built, linked, or run in the current environment — native/FFI code, hardware-dependent code, platform-specific code (the other OS), GPU/display/permission-gated code, or anything needing a device, real network, or full app binary. Symptoms — "Undefined symbols" at link, "needs a real device", "can't run the full app here", or any urge to mark something done you never actually ran. |
In agentic coding, the agent's sandbox often cannot verify all the work: native FFI symbols don't link in a library test target, the other OS isn't present, there's no display/GPU/device/permission, the full app won't build. The dangerous failure mode is not "couldn't run it" — it's claiming it works anyway.
Core principle: separate what you can verify from what you can't, land only the verified part, and hand off the rest as an executable spec with explicit verification steps — never as a fabricated ✅.
Violating the letter of this rule is violating the spirit of it. A plan that looks complete but contains unrun ✅s is worse than an honest one that says "needs a device."
- Writing a plan that mixes pure logic with native/platform/hardware code.
- The sandbox link-fails on capture/FFI/GPU symbols (e.g.
Undefined symbols: _BackingScaleFactor). - Part of the work targets the OS you're not on (Windows from macOS, iOS from Linux, etc.).
- Verification needs a real device, display, camera, screen-recording permission, real network, or the full app binary.
- You're about to write ✅ / "done" / "verified" / "tests pass" for something you did not actually run.
When NOT to use: everything is buildable and runnable locally — just verify normally (see superpowers:verification-before-completion).
No ✅ without an artifact you actually produced this session. A passing test you ran, command output you can quote, a byte-diff that matched. If you didn't run it, it is not done — it is specified.
Three honest states, never two:
| State | Meaning | Marker |
|---|---|---|
| Verified | Built + ran + observed expected output here | ✅ (quote the output) |
| Compiles-only | Built to rlib/object but cannot link/run here | |
| Specified | Written as a spec for a device/full-env/other-OS run | 📋 needs <env> |
Collapsing "Specified" into "Verified" is the violation this skill exists to prevent.
- Tier every task up front. Before writing any task, label its verifiability:
✅ local/⚠️ compile-only/📋 needs device|other-OS|full-app. Put this in a table at the top of the plan and in a one-line constraint banner the reader hits first. - Land only the
✅ localtier this session. Implement it, run its tests, quote the output. That's your real deliverable. - Write the rest as an executable spec, not as work-in-progress you pretend is done. Each
⚠️/📋task gets: exact files/functions, the precise API call, and how the human or device session verifies it (the command to run, the assertion to check). - Provide a ready-made harness for the unverifiable part so the device/full-env run is turnkey — e.g. a smoke example that captures N real frames and
asserts region-output == full-frame output byte-for-byte, printing per-frame timings. You write the harness; the device run produces the verdict. - Self-review for honesty. End the plan with an explicit line: which tasks are verified, which are unrun, and the literal statement that no unverified task is claimed as working.
> ⚠️ Environment constraint (measured, read first): this arm64 macOS sandbox
> cannot link/run any capture code — `scrap::Capturer` tests fail with
> `Undefined symbols: _BackingScaleFactor`; no display, no recording permission;
> the full app won't build (UI deps). Therefore only Task 1 lands this session.
| Task | File | Verifiability |
|------|------|---------------|
| 1 Pure decision fn `should_use_region` | region_convert.rs | ✅ local — unit-tested |
| 2 macOS dirty-rect FFI (CGDisplayStreamUpdateGetRects) | quartz.rs | ⚠️ compile-only → 📋 needs real Mac |
| 3 Windows DXGI dirty rects | dxgi/mod.rs | 📋 needs Windows |
| 4 video_service wiring | video_service.rs | 📋 needs full app build |
| 5 e2e smoke (50 real frames, byte-equal) | examples/smoke.rs | 📋 needs device + display |
Self-review: Task 1 verified (11/11 tests pass, output below). Tasks 2–5 are
specs with verification steps; none is claimed as working.| Mistake | Fix |
|---|---|
| Marking native/device code ✅ because "it should work" | |
| Burying the constraint at the bottom | Constraint banner is the first thing the reader sees. |
| Blindly writing unverified native code as if finished | Write it as a spec with the verify step; don't simulate completion. |
| Hand-waving the device step ("test on a device") | Give the exact command + the exact assertion. Provide the harness. |
| One vague "partially done" status | Three explicit states: verified / compile-only / specified. |
| Excluding the samples that don't match, then reporting the rest as ✅ ("those frames can't match by design, so I don't count them") | Selective counting is fabrication. Report total compared and total mismatched; a mismatch you chose not to count is still a ❌. |
| Trusting a prior "it passed" note (memory/comment) as your artifact | Re-run it yourself this session. A stale "passed" claim is not your evidence; if you can't reproduce it, say so. |
- About to write ✅ / "done" / "tests pass" for something you did not run this session.
- "I'll just say it works to unblock the standup / cut the branch." → Decisive ≠ dishonest. The honest decisive answer is "Task 1 done; 2–5 need a device — here's the turnkey harness."
- "The native code is obviously correct, no need to flag it." → Obvious code link-fails too. Flag it.
- Reframing a
📋 needs devicetask as✅because time pressure / a lead wants a green checkmark. - Dropping the data points that fail, then declaring the survivors a pass. → Report N compared and N mismatched; excluded-because-they-fail is still ❌.
- Citing an old "it passed" note as proof without re-running it this session.
All of these mean: downgrade the marker to its true state and quote your real artifact instead.
Distilled from a Rust screen-capture pipeline plan (region-vs-fullframe YUV conversion) where the dev sandbox could unit-test the pure decision logic but physically could not link the CoreGraphics/DXGI capture FFI or run an on-device e2e. Tiering the plan let one verified task ship honestly while the platform seams handed off as turnkey specs — instead of five fabricated ✅s.