Skip to content
Merged
27 changes: 18 additions & 9 deletions .claude/skills/predict-audit/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
name: predict-audit
description: Deep, multi-lens security & correctness audit of the DeepBook Predict smart contracts (the predict + propbook + block_scholes_oracle + account Move source). Use when asked to audit Predict, hunt bugs/invariant-violations/exploits in Predict, review Predict contract changes before merge, or assess economic safety of the Predict protocol. Smart-contract audit — finds bugs, invariant violations, exploits, and architecture/maintainability issues; it does not check deploy/ops readiness. RUNNING IT IS EXPENSIVE (a full run is hundreds of subagents / up to ~100M tokens) and REQUIRES explicit user confirmation before every run — never execute it automatically.
description: Deep, multi-lens security & correctness audit of the DeepBook Predict smart contracts (the predict + propbook + block_scholes_oracle + account Move source). Use when asked to audit Predict, hunt bugs/invariant-violations/exploits in Predict, review Predict contract changes before merge, or assess economic safety of the Predict protocol. Smart-contract audit — finds bugs, invariant violations, exploits, and architecture/maintainability issues; it does not check deploy/ops readiness. RUNNING IT IS EXPENSIVE (a full run is a few hundred subagents / millions of tokens — each harness now caps its own agent count by construction) and REQUIRES explicit user confirmation before every run — never execute it automatically.
---

# Predict Smart-Contract Audit

## ⛔ STOP — explicit run confirmation is MANDATORY
This skill launches **expensive multi-agent audits** — a maximal end-to-end run is **hundreds of subagents and up to ~100M tokens**. It must NEVER run automatically or as a side effect of being invoked, read, or mentioned.
This skill launches **expensive multi-agent audits** — a maximal end-to-end run (all three harnesses) is **a few hundred subagents and several million tokens**. Each harness now BOUNDS its own agent count by construction (`maxRounds`×units + `verifyCap` verify, severity-gated), so it cannot run away to the old ~100M / 1000-agent-cap territory unless you explicitly raise the caps via args. It must NEVER run automatically or as a side effect of being invoked, read, or mentioned.

**Before launching ANY harness run (`orchestrator` / `ownership-walk` / `rule-sweep`), you MUST:**
1. Show the user the run plan: which harness(es), the scope (`lenses` / `units` / `rules`, or "full"), `dryRounds`, and the budget / rough cost (agent count, token estimate).
1. Show the user the run plan as **two independent axes** plus cost: (a) **breadth/scope** — which harness(es) and `lenses` / `units` / `rules` (or "full"); (b) **depth tier** — `depth: low|standard|max` (see the table under "Launching a run"); and the budget / rough cost (agent count, token estimate). Splitting breadth from depth lets the user pick e.g. "full breadth, low depth" (all lenses, `depth:'low'` = single round) for a cheap complete pass and reserve `depth:'max'` for runs with ample token budget. When you ask the launch questionnaire, ask **both** axes.
2. Ask explicitly — e.g. *"Run this audit? (≈N agents / ≈X tokens)"* — and **WAIT for an explicit "yes."**
3. Only after the user confirms, call `Workflow(...)`. If anything is ambiguous, ask; never assume.

Expand Down Expand Up @@ -41,13 +41,13 @@ fixed_math (math) ─────────────┐ │
## Method (5 phases)
1. **Ground truth — MAIN LOOP ONLY.** `sui build` + `sui test` all four packages (`--warnings-are-errors`), and a localnet sim smoke. "Tests don't compile / are red" is itself a finding. Results feed the audit.
2. **Loop-until-dry find.** Each lens re-samples across rounds (told what's already found, so it hunts new ground); the union grows until K consecutive dry rounds (`dryRounds`) or the token-budget floor — converting LLM sampling into near-enumeration and auto-retrying flaky units.
3. **Adversarial verify.** Every candidate finding is independently refuted/confirmed; High/Critical get an empirical-repro attempt.
3. **Adversarial verify (cross-model).** Every candidate finding is independently refuted/confirmed; High/Critical get an empirical-repro attempt. In the lens fan-out, verify is **cross-model for bias reduction** — refute + repro run on **codex**, the settled-decision check on **Claude** — so each escalated finding clears two different models (needs the codex CLI; degrades to Claude-only if codex is absent).
4. **Empirical deep pass (required).** The economic-simulation lens runs the localnet + Python sims and writes new adversarial scenarios to actually break invariants.
5. **Consolidate (no-slip) + synthesize.** Read each harness's FULL output file (the notification preview is truncated), run `consolidate.py` to deterministically merge all three into ONE `consolidated-report.md` covering **open + settled + refuted + coverage** with a `DROPPED 0` accounting guarantee, then write the executive summary on top. The orchestrator's promote pass + the consolidator's coverage section catch buried items; never hand-pick from a truncated return.

## Harnesses
Three complementary review *shapes*, all sharing this `primer.md`, the report format, and the read-only / compiler-in-main-loop discipline. Read `primer.md` first either way.
- **Lens fan-out** — `orchestrator.workflow.js`: the 10 perspective lenses (broad multi-angle bug hunt). Or hand a single `lenses/NN-*.md` to one session. Scope with `args.lenses`.
- **Lens fan-out** — `orchestrator.workflow.js`: the 10 perspective lenses (broad multi-angle bug hunt). Or hand a single `lenses/NN-*.md` to one session. Scope with `args.lenses`. Two refinements live here (not yet ported to the siblings): **per-lane retirement** — a lens that goes dry for `dryRounds` rounds drops out, so later rounds re-run only the lenses still surfacing issues (set `dryRounds:1` to re-run only lenses fresh last round, `maxRounds:1` for a single pass) — and the **cross-model verify panel** (codex refute+repro, Claude settled).
- **Ownership walk** — `ownership-walk.workflow.js`: recursive *per-module* conformance against the contextual ownership/boundary/policy rules in `references/ownership-rules.md` (R1–R7). Map (barrier) → per-module check → adversarial verify; the unit of work is a code node, not a perspective. Scope with `args.units` (subsystem clusters). Use this for "who owns what / where are boundaries violated."
- **Rule sweep** *(refreshed from the old root `rule-auditor.md`)* — `rule-sweep.workflow.js`: per-*rule* sweep for the 11 mechanical/local rule families (config-reads, config API shape, public-API exposure, object-identity keys, `create_and_share` naming, ProtocolConfig gate matrix, arithmetic-guard noise, test-coverage, timestamp semantics, events hygiene, **dead-field liveness** — the exhaustive write-only / read-only-mirror sweep that catches the rebate-reserve bug class). Broad-shallow is correct there; the contextual ownership families (ex-rule-auditor 6/8/9) live in the ownership walk instead. Scope with `args.rules`.

Expand All @@ -60,7 +60,7 @@ sui move build --path packages/<pkg> --warnings-are-errors # pkg ∈ predict p
sui move test --path packages/<pkg> --gas-limit 100000000000 # each of the four (predict is the big suite)
(cd packages/predict/simulations && bash run.sh --python-only) # sim smoke; localnet `bash run.sh` is also main-loop-only
```
**Step 2 — launch a harness.** Each **loops-until-dry** (tune `dryRounds`/`maxRounds`). Precede the turn with a budget directive (e.g. `+100m`) to cap a maximal full run — it loops until ~70% spent, reserving 30% for verify. Scope with `lenses`/`units`/`rules`; the config log echoes scope + budget. The no-arg default loops the WHOLE fleet until dry (= maximal mode):
**Step 2 — launch a harness.** Each **loops-until-dry**, but is **BOUNDED BY CONSTRUCTION**: defaults are `maxRounds 3` / `dryRounds 2` / `verifyCap 60`, and verify is severity-gated (Info/Low/cleanup reported raw, no subagent; Medium 1 verifier; High/Critical the full panel). So total agents ≤ `maxRounds`×lenses/units/families + `verifyCap`×panel — it cannot hit the 1000-agent cap. **Do NOT rely on a `+NNNm` budget directive to bound it** — the budget often does NOT propagate into a background workflow (`budget.total` arrives null), which is exactly how the first maximal run blew past 25M tokens to the agent cap. Deepen a run by raising `maxRounds`/`verifyCap`/`maxFindings` via args, not via the budget. Scope with `lenses`/`units`/`rules`; the config log echoes scope + caps. The no-arg default runs the whole fleet at the bounded defaults:
```
Workflow({ scriptPath: '.claude/skills/predict-audit/orchestrator.workflow.js',
args: { groundTruth: '<build/test summary>', scope: 'full protocol at HEAD', lenses: ['invariants','adversarial'], maxFindings: 8 } })
Expand All @@ -69,6 +69,14 @@ Workflow({ scriptPath: '.claude/skills/predict-audit/ownership-walk.workflow.js'
Workflow({ scriptPath: '.claude/skills/predict-audit/rule-sweep.workflow.js',
args: { groundTruth: '<build/test summary>', rules: ['events-hygiene'], maxFindings: 8 } })
```
**Depth tiers** — pass `depth` to the **orchestrator** (orthogonal to breadth; `low` still runs ALL lenses, just fewer rounds). An explicit `maxRounds`/`verifyCap`/`maxFindings` arg overrides the preset. The `ownership-walk` / `rule-sweep` siblings don't read `depth` yet — set the raw caps directly.

| `depth` | rounds / dry | verifyCap | use when |
|---|---|---|---|
| `low` | `maxRounds 1` | 30 | **full breadth, low depth** — a cheap complete pass, one sample per lens |
| `standard` *(default)* | `maxRounds 3` / `dryRounds 2` | 60 | the bounded default |
| `max` | `maxRounds 5` / `dryRounds 3` | 100 (+ `maxFindings 16`) | reserve for special runs with ample token budget |

**Step 3 — consolidate in the MAIN LOOP (no-slip guarantee).** Each harness's FULL result is persisted to its task output file — **the notification preview is TRUNCATED; never synthesize from it.** Re-run any failed units first (`Workflow({ scriptPath, resumeFromRunId })`), then run the deterministic consolidator over the full files:
```
python3 .claude/skills/predict-audit/consolidate.py .claude/predict-review/reports/<run>/ <orchestrator.output> <ownership-walk.output> <rule-sweep.output>
Expand All @@ -84,7 +92,7 @@ python3 .claude/skills/predict-audit/track.py .claude/predict-review/ merge --ru
### Which harness, in what order
- **"Audit Predict" / find bugs** → start with the **lens fan-out** (orchestrator). For a pre-merge pass, add the **ownership walk** + **rule sweep**.
- **Subsystem deep-dive** → ownership walk on that `units` cluster. **Mechanical hygiene** → rule sweep on the relevant `rules`.
- **Maximal end-to-end:** ground truth → run the three harnesses loop-until-dry, **each in its OWN turn with its OWN budget directive** — they share the turn's `budget` pool, so launching all three in one turn starves the later two; give each e.g. a `+35m` → `consolidate.py` → `track.py`. Last-line-of-defense mode, hundreds of agents. Quick pass instead: scope + `dryRounds: 1`.
- **Maximal end-to-end:** ground truth → run the three harnesses (each bounded by its default caps) → `consolidate.py` → `track.py`. Run them in **separate turns** (so a slow one doesn't block the others, and to read each full output before the next), but you no longer need a per-turn `+NNNm` budget directive — the caps bound each run regardless. Last-line-of-defense mode, a few hundred agents total. For more depth, raise `maxRounds`/`verifyCap` via args; for a quick pass, scope (`lenses`/`units`/`rules`) + `maxRounds: 1`.

## Hard rules (non-negotiable)
- **Read-only on source.** Never modify `packages/*/sources/**`. The only writes are reports (to `.claude/predict-review/reports/<date>/`, gitignored) and temp sims/scripts (to the session scratchpad).
Expand Down Expand Up @@ -114,5 +122,6 @@ python3 .claude/skills/predict-audit/track.py .claude/predict-review/ merge --ru

## Operational notes (read before relying on a single run)
- **StructuredOutput flakiness.** A small fraction (~5–15%) of find/check/sweep agents can exhaust the StructuredOutput retry cap on the most verbose units and return nothing (the run reports them under `failures`, and the config log shows what ran). Mitigated by concise-candidate output + proof-in-verify + lower effort, but not eliminated. **Recovery:** `Workflow({scriptPath, resumeFromRunId})` re-runs only the failed units — successful agents return cached — so a partial run is cheaply completed.
- **Run-to-run variance is handled in-harness (loop-until-dry).** LLM finding samples the space rather than enumerating it, so all three harnesses re-run each lens/module/rule across rounds and union until K consecutive dry rounds (`dryRounds`, default 2–3) or the budget floor. Tune thoroughness with `dryRounds`/`maxRounds` and the turn budget — e.g. a **`+100m`** directive makes a run keep going until ~70% spent (30% reserved for verify). A converged run is a strong floor; still treat it as a floor, not a proof of absence. (`resumeFromRunId` still tops up any units a run didn't reach.)
- **Cost.** A full lens or ownership-walk run is dozens of agents / millions of tokens. Always scope first (`args.lenses` / `args.units` / `args.rules`) and check the config log; the no-arg default runs the whole fleet.
- **Run-to-run variance is handled in-harness (loop-until-dry).** LLM finding samples the space rather than enumerating it, so all three harnesses re-run each lens/module/rule across rounds and union until K consecutive dry rounds (`dryRounds`, default 2) or the round cap (`maxRounds`, default 3). (The lens fan-out now applies `dryRounds` **per-lane** — a converged lens retires individually instead of waiting on one global counter.) Tune thoroughness with `maxRounds`/`verifyCap`/`maxFindings` via args — **not** the turn budget: the `+NNNm` directive often does NOT propagate into a background workflow (`budget.total` arrives null), so it's an unreliable bound. A converged run is a strong floor; still treat it as a floor, not a proof of absence. (`resumeFromRunId` still tops up any units a run didn't reach.)
- **Cost is bounded by construction.** Each run's agents ≤ `maxRounds`×(lenses/units/families) + `verifyCap`×panel, and verify is severity-gated (Info/Low/cleanup reported raw, no subagent). At defaults that's well under the 1000-agent cap / a few million tokens. Still scope first (`args.lenses` / `args.units` / `args.rules`) for a cheaper run; the no-arg default runs the whole fleet at the bounded defaults.
- **Cross-model verify needs the codex CLI (lens fan-out only).** The orchestrator routes the refute + repro verifiers to the `codex:codex-rescue` agent — a different model than the Claude finder — for bias reduction; the settled-decision verifier stays on Claude. Spike-verified: codex emits schema-valid `VERDICT_SCHEMA` JSON and re-derives findings from its own git/grep evidence. If the codex CLI is unavailable those verdicts return `null` and `aggregate()` falls back to the Claude verdict(s) (a lone Medium → `uncertain`). Codex prompts are kept tightly scoped (it can balloon/hang on open-ended reads). NOTE: codex reasoning effort is governed by codex config, **not** the workflow `effort` knob — confirm it runs at the intended tier when wiring. `rule-sweep` / `ownership-walk` are still Claude-only (a planned follow-up).
Loading
Loading