Skip to content

002-wm-prior-rollout.md

Latest commit

 

History

History
335 lines (291 loc) · 23.8 KB

002-wm-prior-rollout.md

File metadata and controls

335 lines (291 loc) · 23.8 KB
Raw

002 — World-Model PRIOR transfer into the K-step ROLLOUT fit (Gate 001 re-open lever #2)

Status: STOP (2026-07-19). All three STOP conditions trigger at the 3-seed median. Rung: Phase-D — same carried object as Gate 001 (the fitted W1 world-model weights as the warm-start PRIOR for a target world's model at small transition budgets), but measured under the K-step rollout objective and ruler — the objective whose consumer is real. Gate 001 (one-step fit, one-step ruler) closed PARTIAL-downgraded: average warm−cold margin ~0 (+0.012), specificity retracted as noise. Route A separately proved one-step accuracy and rollout usefulness are DECOUPLED (changed 0.0→0.93 while top-1 regret worsened 1.0×→5.6×). So Gate 001's ~0 does not answer this gate's question; this is the registered re-open lever #2 (ROADMAP "re-open levers"), not a re-run.

Hypothesis

Fitting a target world's model through the Route-A K-step rollout curriculum with world W1's one-step-fitted weights replacing the target-data one-step warm-start stage reaches a higher held-out K-step rollout changed-cell accuracy than the full cold curriculum at the same small transition budget, the advantage is largest at the smallest budget, and it is attributable to the content of the W1 weights (neither a scrambled-W1 start nor the near-identity seed pushed through the same rollout-only schedule reproduces it).

Metric + consumer

  • Primary metric — RA64. rollout_accuracy_curve (src/world_model.mojo:1027) evaluated on the held-out pool: per-tick changed-cell accuracy of a 64-tick autoregressive dream against the ground-truth episode (identity predictor scores 0 at every tick by construction), averaged over all 64 ticks and all 8 held-out episodes → one scalar per fitted vector. This is the currency of the actual consumer — test_dream_rank's 64-tick dream — not the one-step proxy Route A proved decoupled from it.
  • Held-out currency. Fit on B training transitions; score RA64 on a fixed held-out pool of 512 transitions = 8 whole 64-tick episodes per target world, collected by the same collector, disjoint from every training subset, identical across all arms, budgets and reps within a seed. rollout_accuracy_curve requires whole-episode layout; 512/64 = 8 episodes satisfies it. No per-arm held-out construction.
  • Diagnostics (reported, never gated): one-step held_out_score changed (continuity with Gate 001 — does the rollout stage cost one-step accuracy?); held_out_score_split wall/open at room B=32 (locality context); RA8 (mean of curve ticks 0–7, the fit's own K=8 horizon); RAWPRIOR = RA64 of the raw, un-fine-tuned w1 on each target world (zero target fitting, one scoring call per world×seed) and the derived FT = median_r RA64_warm,r − RAWPRIOR (what the target fit adds over the naked prior — see the reinterpretation clause under PARTIAL).
  • Consumer. The GO/STOP decision green-lights (or kills) Phase-D Increment 2 — the Reptile meta-learned WM prior across a family of worlds (ued.mojo layouts), measured on a held-out layout. The named downstream wire: a GO obliges a follow-up re-read of the booked negative tests/test_dream_rank.mojo (room top-1 regret 4.64×/2.39× vs the 2.0 bar, bars unchanged at MIN_W2_TAU=0.5 / MAX_REGRET=2.0) with the warm-fitted room model — that re-read is context, not a bar of this gate: the gate stays in model currency (T-POC-2's lesson — a better model does not automatically fix picking).

Setup

  • The prior. w1 = W1 (base world) fit once per seed by fit_wm_restarts at PRIOR_BUDGET=128, running off seed(SEED) exactly as tests/test_wm_prior.mojo does — so the prior object is bit-identical to Gate 001's at each seed. It is the one-step-fitted vector, NOT a rollout-fine-tuned W1: Route A measured W1's consumer calibration worsening (5.6×→19.4×) under the rollout fine-tune, so carrying the rollout-fitted W1 would confound "prior transfer failed" with "prior was damaged before shipping". (The rollout-fitted-W1 variant is registered out of scope; see re-open levers at the end.) Sanity bar (smoke, not a verdict): W1 train changed ≥ 0.6 (WM_ACCEPT; Gate 001 observed 0.93/0.80/0.90). Report W1's RA64 on its own held-out episodes as scale context for the ruler.
  • Worlds × budgets. w ∈ {columns, room}, B ∈ {32, 128} transitions. Room is the crux (density 0.0027 — Route A's regret miss at both seeds, the world a data-starved K-step fit should need a prior most); columns corroborates. B=64 is dropped relative to Gate 001 to pay for the fourth arm (the data-efficiency contrast needs exactly {32,128}; Gate 001's B=64 column never carried a bar). At stride 4 / K=8, B=32 yields 7 overlapping rollout windows, B=128 yields 30 (wm_rollout_cases, src/world_model.mojo:873) — thin but nonzero at B=32, which is the point.
  • The fixed schedule (byte-identical wherever a stage runs; nothing tuned per arm/world/ budget/rep): Stage 0 = Gate 001's fit_arm one-step recipe (two stages, σ0.3→0.1/750 + σ0.08→0.02/500, N=16, reg_lambda=0, single fit, no restart loop); rollout Stages 1+2 = the Route-A pinned constants exactly (src/adapt.mojo:732–747: 250 iters α0.12→0.05 σ0.15→0.06, then 150 iters α0.04→0.01 σ0.05→0.02, N=16, stride 4, K=8, reg_lambda=0).
  • Four arms — each rep runs all four; they differ only in what happens before the rollout stages:
    • COLD — the full curriculum on target data: WeightedWMMemory.seed → Stage 0 one-step fit on the B transitions → rollout Stages 1+2 on the same B transitions' windows.
    • WARM — copy_weights(fast, w1); Stage 0 is skipped (the prior IS the stage-0 point) → rollout Stages 1+2. WARM spends strictly less compute than COLD by construction — the conservative direction: a GO cannot be bought with extra iterations, and "respending" Stage 0's iters would mean inventing a WARM-only schedule (anti-stone-soup).
    • SCRAMBLED — full Fisher-Yates permutation of w1 (per-cell, seed-fixed, never hand-chosen) → rollout Stages 1+2. Controls value-distribution/scale: "any warm blob".
    • IDSEED — WeightedWMMemory.seed (the canonical near-identity init, i.e. COLD's own starting point) → rollout Stages 1+2, Stage 0 skipped. The compute-matched refuting control: identical schedule and compute to WARM, differing only in vector contents. It separates "the prior's carried knowledge helps" from "skipping the one-step stage happens to help the rollout objective" — the likeliest fake-GO, which SCRAMBLED alone cannot catch (a permuted vector is a pathological init, so scrambled can fail for reasons unrelated to the stage-skip question).
  • Safety net DISABLED in all measured arms. fit_wm_rollout_restarts's keep-better-of {pre, post} by train rollout mean (src/adapt.mojo:796–817) is train-side model selection between two candidates with asymmetric fallback quality across arms (WARM's "pre" would be the full-budget W1 prior, COLD's a B-budget fit) — inside this instrument it is scaffolding. Rep-medians (below) replace its robustness role. Likewise the fit_wm_restarts restart loop runs only in the shared W1-prior setup, never in a measured arm.
  • Instrument (mandatory house methodology since Gate 001's amendment — bimodal-per-fit metrics get R=5 paired fits + CRN). Each (world × budget × arm) cell = median over R=5 reps; per-seed margins = median over the 5 paired per-rep differences. RNG contract, extending tests/test_wm_prior.mojo:128–137 with one new reseed class: data / perm / fit seeds as in Gate 001 (disjoint derived ranges), plus fit2_seed(w,b,r) reseeded immediately before the rollout stages in every arm — because COLD's Stage 0 consumes RNG the other arms don't, a single pre-fit reseed would leave the rollout stages of COLD on a shifted stream; the second reseed restores exact CRN pairing of the rollout-stage perturbation stream across all four arms at rep r. fit_seed(w,b,r) still precedes Stage 0 (COLD only consumes it).
  • Driver: new tests/test_wm_prior_rollout.mojo (# suite-tier: full), measurement-only (prints margins + provisional per-seed bars; hard-raises only on smoke), same shape as test_wm_prior.mojo.

Define, per seed, at the paired rep-medians (RA64 currency throughout):

  • margin MK_w(B) = median_r[RA64_warm,r − RA64_cold,r]
  • specificity SK_w(B) = median_r[RA64_warm,r − RA64_scrambled,r]
  • content-over-schedule CK_w(B) = median_r[RA64_warm,r − RA64_idseed,r]
  • data-efficiency = median_r MK_room(32) − median_r MK_room(128) (reps don't pair across budgets)

Cross-seed aggregation: medians over seeds; the sign bar is an AND across the primary seeds.

GO / STOP / PARTIAL thresholds (committed now)

RA64 averages 64 ticks × 8 episodes, so it is far less bimodal than Gate 001's one-step changed; bars are set accordingly smaller and were fixed before any number was seen.

GO — all five must hold (room, 3-seed medians of paired rep-medians):

  1. Prior helps under the rollout objective at low data: MK_room(32) ≥ +0.10.
  2. Data-efficiency signature: MK_room(32) − MK_room(128) ≥ +0.05.
  3. Prior-specificity: SK_room(32) ≥ +0.05.
  4. Content, not curriculum shape: CK_room(32) ≥ +0.05 (warm beats the compute-matched near-identity control).
  5. Sign of MK_room(32) agrees on both primary seeds (0 and 1); seed 2 is the tiebreak.

STOP — any one triggers (negative result; the prior axis closes for this ladder):

  1. No rollout-transfer either: MK_room(32) ≤ 0 at the median. Combined with Gate 001's clean ~0 under the one-step objective, both registered objectives then read "prior not better than cold" — the W1-prior lever is closed for Increment 2 (only the insurance secondary, below, could survive on its own narrower terms).
  2. The stage-skip, not the prior: CK_room(32) ≤ 0 and CK_room(128) ≤ 0 — the near-identity init pushed through the identical rollout-only schedule does as well as w1; any warm−cold margin is curriculum shape, not carried knowledge.
  3. Any-warm-blob: SK_room(32) ≤ +0.02 and SK_room(128) ≤ +0.02 — scrambled matches warm; the effect is not W1's structure.

PARTIAL — real signal missing at least one GO bar without tripping STOP, e.g.:

  • Columns clears (MK_col(32) ≥ +0.10, CK_col(32) ≥ +0.05) but room stays flat (0 < MK_room(32) < 0.10) — the world Route A's rollout fix already reached responds; the data-starved crux still resists. Book it; not a GO.
  • Bars 1/3/4 pass but the data-efficiency signature fails — the prior helps but not demonstrably via data-efficiency.
  • Bars pass at the median but the primary seeds split in sign (bar 5) — add the seed-2 tiebreak and re-read; if still split, PARTIAL.
  • Reinterpretation clause (registered now): if warm wins but FT_room(32) ≤ 0 — i.e. the raw, never-fine-tuned w1 already scores ≥ the warm arm's median RA64 — the correct booking is "W1's model already covers the target at the rollout horizon": a cheaper and different claim than prior→fit transfer, and the "carry the prior into the fit" hypothesis is NOT supported even though warm > cold. Book PARTIAL with that finding named; a GO additionally requires FT_room(32) > 0.

Secondary hypothesis (independent bars) — "prior as insurance", registered per Gate 001's close

Gate 001's surviving conditional: when the cold fit degenerates on a low-budget draw, the prior rescues it wholesale. Registered here as a failure-rate claim (the metric Gate 001's close prescribed), scored over the 15 (seed × rep) room B=32 measurements per arm (3 seeds × 5 reps), nearly free from the per-rep printouts:

  • Degenerate rep: held-out RA64 < 0.05 (identity collapse scores exactly 0).
  • F_arm = fraction of the 15 (seed, rep) cells degenerate.
  • INS-GO: F_cold ≥ 3/15 (testable) and F_warm ≤ F_cold − 3/15 and F_warm ≤ F_scrambled − 3/15 (the rescue exists and is prior-specific).
  • INS-STOP: F_cold ≥ 3/15 and F_warm ≥ F_cold (failures exist; the prior does not reduce them — insurance refuted under the rollout curriculum).
  • INS-NOT-TESTABLE: F_cold ≤ 1/15 — the rollout curriculum's cold arm rarely fails at this budget; nothing to insure. Booked as untestable-here, not as a pass. These bars share no statistic with the primary (failure counts vs median margins); INS-GO with a primary STOP is a legal, meaningful outcome (insurance without average advantage) and licenses at most an insurance-shaped next rung, never Increment 2.

Seeds

Primary: 0 and 1, both worlds. Seed 2, room only, as tiebreak (house convention) — and required regardless of tie for the insurance denominator (seed 2 is Gate 001's known degenerate-cold draw; excluding it would be selection against the secondary hypothesis's only known positive instance — so it runs unconditionally). Medians for all magnitudes; the sign bar is an AND across seeds 0 and 1.

Refuting control

IDSEED is the load-bearing one (STOP #2): identical rollout-only schedule and compute to WARM, differing only in vector contents — if it matches warm, the "prior" added nothing over the curriculum truncation itself. SCRAMBLED (STOP #3) additionally rules out value-scale effects. COLD (STOP #1) is the deployment alternative the prior must actually beat. If none of the three can produce a "no", the run is broken, not blessed — all three STOPs are live by construction.

Anti-stone-soup clause

Any of the following makes a later pass a documented negative, not a GO:

  • per-arm, per-world, per-budget, or per-rep tuning of any schedule constant (iters, alpha, sigma, N, reg_lambda, WM_ROLLOUT_STRIDE, WM_ROLLOUT_K) — the rollout stages run the pinned src/adapt.mojo:732–747 constants everywhere or nowhere;
  • respending COLD's Stage-0 iterations in any warm arm (extra rollout iterations, extra stages, a longer anneal) — the registered compute asymmetry is one-directional: WARM ≤ COLD;
  • re-enabling the train-side safety net (or any keep-better-of / restarts-and-keep-best) inside a measured arm — including "just for COLD because production does it": the arms run bare fits, symmetrically, and R=5 medians are the only robustness mechanism;
  • any selection among reps other than the median (dropping a "bad" rep = keep-best by another name); any reseeding scheme that de-pairs the arms at rep r, or "fixing" a rep whose CRN stream produced an ugly draw;
  • rollout-fine-tuning w1 before carrying it, re-fitting it per target world, or hand-selecting which W1 weights carry; choosing the scrambled permutation to be gentle;
  • per-arm/per-rep held-out pools, train/held-out overlap, or scoring RA64 on training episodes;
  • moving the degeneracy threshold or the insurance denominators after seeing failure counts.

Mechanism reuse only: fit_operator warm-start buffer, WeightedWMMemory/WMRolloutMemory, wm_rollout_cases, rollout_accuracy_curve, held_out_score(_split) — all existing. The sole new code is the measurement driver tests/test_wm_prior_rollout.mojo. No new learning machinery.

Compute plan (stated; no trimming — user decision 2026-07-19)

Per (world, budget) cell and rep: COLD ≈ 1.6× a Gate-001 fit (one-step 1250×16×B forwards + rollout 400×16×windows×8); each rollout-only arm ≈ 0.6×. Four arms × 5 reps ≈ 17 Gate-001 fit-units per cell vs Gate 001's 15 — but only 4 cells/seed (2 worlds × 2 budgets) vs Gate 001's 6, so the per-seed run is ≈ 0.75× the Gate-001 re-measure, which was tractable. RA64 scoring adds 512 forwards per fit (negligible). No trimming (user decision, 2026-07-19): all cells run regardless of wall clock — reps, seeds, arms, worlds, and budgets are all non-negotiable; if the run overruns, it simply takes longer.

Registered deviations from production (honest flags, committed now)

  1. COLD's Stage 0 is a single one-step fit, not fit_wm_restarts's best-of-4-restarts — restarts are keep-best selection with data-dependent compute; R=5 rep-medians replace the lottery. Consequence: COLD here is a floor on the production curriculum; a warm win over this COLD does not automatically beat production-with-restarts. If GO, a one-off confirmation against restart-enabled COLD at room B=32 is the named follow-up before Increment 2.
  2. The safety net is disabled (above). Consequence: any arm can net-regress its own start; RAWPRIOR/FT exist so that outcome is visible, not hidden.
  3. B=64 dropped relative to Gate 001 (funds the IDSEED arm).

Out-of-scope / re-open levers (named now so they aren't invented post-hoc)

  • The rollout-fitted-W1 prior variant (carry fit_wm_rollout_restarts(W1)'s vector).
  • The small-anchor variant (reg_lambda > 0 toward w1) — still untested since Gate 001.
  • The dream-rank re-read (tests/test_dream_rank.mojo, bars unmoved) — obligatory context after a GO, never a bar here.
  • Prior-as-Stage-0-init (w1 seeding Stage 0, full curriculum, byte-identical compute) — the weaker graft, worth registering only if this gate STOPs narrowly on #2.

Result — STOP (eval 2026-07-19)

Driver tests/test_wm_prior_rollout.mojo (# suite-tier: full), all three seeds run to completion, driver OK, zero SMOKE errors. W1 prior sanity bar (train changed ≥ 0.6) passed at every seed (0.931 / 0.804 / 0.903) — the priors are real fits, not degenerate, so the transfer test is meaningful. Full verbatim per-rep/per-cell output: scratch/gate002_seed{0,1,2}.txt.

Room (crux) — cross-seed margins, 3-seed medians of the paired rep-medians:

metric seed 0 seed 1 seed 2 median bar pass
MK(32) = warm−cold −0.0267 +0.0020 0.0 0.0 ≥0.10 ✗
MK(128) −0.0052 0.0 0.0 0.0 —
data-eff M(32)−M(128) −0.0215 +0.0020 0.0 0.0 ≥0.05 ✗
SK(32) = warm−scr −0.0257 +0.0010 −0.1279 −0.0257 ≥0.05 ✗
SK(128) −0.0124 −0.0438 −0.1572 −0.0438 —
CK(32) = warm−idseed −0.1112 +0.0073 0.0 0.0 ≥0.05 ✗
CK(128) −0.1061 −0.1200 −0.1143 −0.1143 —
FT(32) = warm−RAWPRIOR −0.0104 0.0 0.0 0.0 >0 (reinterp.) ✗

Columns (corroboration): MK(32) 0.0 / 0.0 / −0.2812 → median 0.0 (no help); CK(32) −0.0836 / +0.0985 / +0.1560 → median +0.0985 (mixed, does not clear the room-anchored decision).

GO (all five required): 1 MK32≥0.10 ✗(0.0) · 2 dataeff≥0.05 ✗(0.0) · 3 SK32≥0.05 ✗(−0.0257) · 4 CK32≥0.05 ✗(0.0) · 5 sign-AND on seeds 0,1 ✗(−/+). GO fails 0/5.

STOP (any one triggers):

  1. No rollout-transfer: MK_room(32) median 0.0 ≤ 0 — TRIPPED.
  2. Stage-skip, not the prior: CK_room(32) median 0.0 ≤ 0 AND CK_room(128) median −0.1143 ≤ 0 — TRIPPED.
  3. Any-warm-blob: SK_room(32) median −0.0257 ≤ 0.02 AND SK_room(128) median −0.0438 ≤ 0.02 — TRIPPED.

All three STOP conditions trigger independently at the median. This is not a borderline call decided by one bar — MK, CK, and SK all point the same way, and CK is frequently negative (warm often worse than the compute-matched near-identity control), meaning the prior's content is not merely neutral under the rollout objective, it is sometimes actively unhelpful. The reinterpretation clause is moot (warm never beat cold, so "W1 already covers the target" doesn't apply either) but is recorded for completeness: FT(32) median 0.0, not > 0.

Refuting controls did their job, in the failing direction. IDSEED (the load-bearing control) shows warm performing on par with or worse than a bare near-identity init pushed through the identical rollout schedule (CK32 median exactly 0.0, CK128 clearly negative) — the compute saved by skipping Stage 0 is not being spent productively on the prior's content. SCRAMBLED shows the same pattern (SK medians negative at both budgets): a scrambled W1 blob does better than the real W1 prior at seed 2 room B=32 in particular (SK32 = −0.1279, i.e. scrambled beats warm by ~0.13) — a genuinely surprising direction the registered controls were built to catch, and did.

Secondary — insurance hypothesis: NOT TESTABLE

Failure-rate over the 15 room-B=32 (seed × rep) cells, degenerate = RA64 < 0.05: F_cold = 0/15, F_warm = 0/15, F_scrambled = 0/15 — every arm's RA64 stayed at or above ~0.08 in every rep of every seed at room B=32 (raw values verified against scratch/gate002_seed{0,1,2}.txt). Per the pre-registered INS-NOT-TESTABLE bar (F_cold ≤ 1/15): not testable here — the K-step rollout curriculum's cold arm never degenerates at this budget, so there is nothing for a prior to insure against. This is itself informative and distinct from Gate 001: the one-step COLD fit had a real degenerate failure at seed 2 (changed 0.0 at 4/5 reps). Under the rollout metric the same seed-2 room draw does not register as degenerate (RA64 stays at 0.08984 across every rep, arm, and budget) — but the honest reason is a floor in the RA64 metric on room, not Stage 0 rescuing the fit: seed 2's own 1step-changed diagnostic is 0.0 in all 5 reps (Stage 0 learned nothing there either), yet 64-tick rollout accuracy still lands at that ~0.09 baseline for every arm including RAWPRIOR. RA64 on room simply doesn't have as low a floor as the one-step metric did, so this instrument at this budget cannot distinguish a "working" cold fit from a failed one by the 0.05 degeneracy threshold. The "prior as insurance" hypothesis is not refuted by this gate — it is simply not exercised at this budget under this metric, and would need either a lower degeneracy threshold calibrated to RA64's actual floor, or a setting where COLD can demonstrably fail on this metric, to be tested at all.

Anti-stone-soup confirmed. Pre-run discipline review (before any seed was run) found the harness clean against the registered design: no fit_wm_rollout_restarts call in any measured arm, rollout-stage constants imported directly from src/adapt.mojo:732-747 (not retyped), the train-side safety net absent from every arm, fit2_seed CRN pairing verified correct across all four arms, held-out pool disjoint from every training subset, zero src/ changes. No deviation from the registered setup occurred during the run.

Durable finding. Under the K-step rollout objective — the objective Route A proved is the one that actually matters for the consumer, and the one Gate 001's one-step result could not speak to — the W1 world-model prior confers no average advantage over cold (MK medians ~0 at both budgets), is not specifically better than a scrambled permutation (SK negative at both budgets), and is not distinguishable from a bare near-identity init run through the same truncated schedule (CK ~0 or negative). Combined with Gate 001's clean ~0 under the one-step objective, the W1-prior-as-warm-start lever is closed for Phase-D Increment 2 under both registered objectives. The registered deviations (COLD's Stage 0 as a single fit rather than best-of-4-restarts; the disabled safety net; B=64 dropped) all point the same direction the gate anticipated — none of them plausibly manufacture a STOP from what would otherwise be a GO; if anything COLD here is a floor on production, so a real effect would have had to be large enough to clear it, and it did not appear. The re-open levers this gate leaves on the table (rollout- fitted-W1 prior, the small-anchor reg_lambda>0 variant, prior-as-Stage-0-init) remain untested, but the specific hypothesis registered here — carrying the one-step W1 prior directly into the rollout curriculum — is refuted, not merely unproven.