superseded.md

Superseded results

This document is a feature of the project, not an embarrassment. Every number below was published, then replaced by a better measurement or a corrected interpretation. Keeping the trail — old number, what replaced it, and why — is what lets a reader trust the numbers that remain. Nothing here was quietly deleted.

Each row: the superseded quantity, where it was published, what supersedes it, and the reason. Filenames in the older rows are pre-restructure paths; see experiments/README.md for where each report lives now. PROJECT_STATE §N and MECHANISM_SPEC.md §N in the "published in" column point at the archived originals of those two documents (/root/residctl-archive/rewritten-sources/); their content was split into findings.md / this file / docs/05-limitations.md and rewritten into docs/02-design.md / docs/design-history.md respectively, but the section numbers refer to the archived text.

superseded	published in	superseded by	reason
All V1 OPT values, fault-count-based rankings, and the MADV_RANDOM "fastest mode" finding	experiments/02-first-harness-superseded.md (marked SUPERSEDED in place, not deleted)	experiments/03-corrected-harness.md	3 defects: circular OPT input (A-1), arms not doing the same work (A-4), wrong primary metric (fault count vs. read_bytes, Defect 3).
Item 10c Sweep 1's framing of "device-busy capped at 0.82-0.86 regardless of --fetch-workers" as evidence about the async architecture's ceiling	experiments/05-async-handler.md	Item 10d's diagnosis (experiments/06-concurrent-demand.md) + item 10e's fix (experiments/07-lookahead-window.md)	The numbers themselves are not wrong, but the INTERPRETATION was: arm D's single-threaded, barrier-free driver structurally could never generate more than one outstanding fetch, so the flat device-busy measured a driver limitation, not an architecture ceiling. Item 10e's Task A re-ran the same style of measurement with real overlap possible and found the SAME flat ceiling persists — see below.
Item 10e Sweep B expectation 2 (concurrently-outstanding rises with window under async)	experiments/07-lookahead-window.md	Campaign 11 Phase 0.1 + Phase 2	Not superseded in the sense of being wrong — item 10e correctly measured concurrently-outstanding=1.00 on arm D. Campaign 11 clarified the SCOPE: arm E (prefetch on) already shows real overlap independent of any window or compute setting (Phase 0.1), so the "1.00" finding is arm-D/prefetch-off-specific, not evidence the mechanism can never overlap. Read item 10e's finding together with Campaign 11 Phase 0/1/2, not in isolation.
Item 10d's retention byte-effect regime claim ("pinned reduces bytes above r_c, costs more below it")	experiments/06-concurrent-demand.md	Item 10e's Sweep C (experiments/07-lookahead-window.md)	Under real cross-chunk overlap (--lookahead-window 1, which item 10d's own barrier-driven driver could not produce), pinned cost MORE bytes than none at every cell but one — the REGIME-BOUNDARY mechanism (stat_pin_broken as indicator) still holds and is not superseded, but the DIRECTION of the effect above r_c reversed. Both reports' raw numbers stand as measurements of the driver architecture they were taken under; only the general claim "retention reduces bytes above r_c" is superseded.
Item 10e's incidental gate-run observation of a 38% wall-clock improvement from --lookahead-window (5.177s→3.227s)	experiments/07-lookahead-window.md (one line in an incidental-observation paragraph)	Campaign 11 Phase 0.2	Traced to a policy confound: the gate script used policy=default (the lowest-index FIFO fallback), never used in any other reported sweep, not policy=layer_order. Sweep B's own layer_order data shows no such effect at any ratio or thread count. The item 10e Task A verification GATE ITSELF (trace/bytes/OPT identity, W=0 regression) is unaffected and remains valid — only the incidental wall-clock observation is superseded.
Every prior report's characterization of prefetch as reducing byte cost only in narrow/no cases (items 10b through 10e, uniformly negative or null on this axis)	experiments/05-async-handler.md, experiments/06-concurrent-demand.md, experiments/07-lookahead-window.md	Campaign 11 Phase 2 + Phase 4	Not contradicted at compute=0 (E still costs more bytes than D everywhere, consistent with every prior report). But under a real compute phase (--compute-ns-per-mib 400000, never tested before this campaign — no prior report includes a compute phase at all), E beats D's read_bytes at r=0.5 and r=0.75, replicated independently in Phase 2 and Phase 4. This does not retroactively invalidate the zero-compute findings — it adds a regime (heavy compute) none of items 10b-10e tested.
Campaign 11 Phase 3's arm A read_bytes column (read_bytes=0 in 11/12 cells) and Phase 4's arm A/B columns (read_bytes=0 in all 5/5 arm-A ratio cells, arm B never measured non-zero either)	experiments/09-chunk-size-sweep.md, experiments/09b-consolidated-6arm-sweep.md	Campaign 12 Phase A's re-run (experiments/09c-baseline-io-repair.md)	Not a host-cache limitation as originally filed — a guest-side drop_caches redirect-ordering bug in both sweep scripts (see §3's corrected entry above). Every D-vs-A and E-vs-A byte comparison in Phase 3/4 was against a baseline that performed no measured I/O; Phase A's re-run supplies real, non-zero arm A/B numbers at both grids using a repaired script and a guard-equipped baseline_main. Everything else in Phase 3/4 (arms C/D/E, OPT) is unaffected and stands.
Campaign 11 Phase 3's characterization "arm D's read_bytes and wall-clock BOTH monotonically increase with chunk size at every ratio" and expectation 5's "wall-clock and bytes move together, no divergence observed"	experiments/09-chunk-size-sweep.md	Campaign 12 Phase B (experiments/09d-chunk-size-floor.md)	Extended, not simply wrong — Phase 3's own tested range ({32,64,128,256} MiB) genuinely showed monotonic, together-moving behavior. Phase B's finer grid ({4,8,16,32} MiB) found neither holds below 32 MiB: read_bytes has an interior floor at 8 MiB for 2 of 3 ratios (not monotonic down to the smallest size), and wall-clock clearly DIVERGES from bytes (wall-clock favors larger chunks, worst at 4 MiB, opposite the byte-minimizing direction). Phase 3's own 32-256 MiB numbers are unaffected and stand as measurements of that coarser range.
The prefetch hit-rate metric's "14-46% ceiling" as evidence of a real, mechanism-level saturation point, cited across items 10b, 10c (A-6's motivation), 10d (A-9's motivation), and item 10e's retention generalization	experiments/04-io-pipelining-diagnostic.md, experiments/05-async-handler.md, experiments/06-concurrent-demand.md, experiments/07-lookahead-window.md	Campaign 13 Phase B (experiments/12-metric-audit.md)	Not a wrong measurement — the recomputed hit-rate numbers reproduce the originally published values exactly (e.g. item 10d's pinned/depth=4/r=0.75 cell: 0.460 against a cited 0.46). What's superseded is the INTERPRETATION: total fetches (demand faults + prefetches), the volume the mechanism actually moves, show no analogous ceiling — item 10b's own r=0.25 cell rises 50% (255→382) across the tested depth range while hit rate stays flat. Three items of mechanism work (A-6, A-9, item 10e's generalization) were evaluated primarily against a metric whose flatness reflects its own denominator, not an independently observable saturation point in the mechanism's real byte-saving performance. The underlying mechanisms' actual wins are NOT invalidated — Campaign 13 Phase B's own Question 1 confirms them directly via total-fetches arithmetic, with zero exceptions across 12 cells.
Campaign 12 Phase D's arm D numbers at 8MiB/r=0.25/c=400000, 8MiB/r=0.375/c=400000, 128MiB/r=0.25/c=400000, 128MiB/r=0.375/c=400000, and 128MiB/r=0.5/c=400000 (the one originally flagged as byte-identical to arm C)	experiments/10-consolidated-sweep.md	Campaign 13 Phase A (experiments/11-policy-determinism.md)	Not wrong as measurements (each is a real n=3 median of real runs) but MISLEADING if read as a stable, reproducible value — Phase A found these 5 cells are genuinely non-deterministic (fault-dispatch-order timing dependence under --driver-threads>1 + --lookahead-window>0 + --compute-ns-per-mib>0 together), with one cell's own n=10 reproduction spanning absent_handled 74-83 out of 80 touches. Campaign 13 Phase C excludes all 5 from every arm-D comparison for this reason. Every other arm D cell in Phase D's grid (15 of 20) is unaffected — confirmed deterministic or not independently flagged. WP1 update: layer_order_declared does NOT fix these — it is non-deterministic under the identical three-factor trigger (WP1 §1.3: 128MiB/r=0.5/c=400000 spans absent_handled 79-90), and at c=400000 reads MORE than layer_order_learned at 5 of 6 tested cells.
Campaign 12 Phase C / Campaign 13 Phase C: "arm D is measurably distant from OPT, D/OPT 1.070-1.775"	experiments/10-consolidated-sweep.md, experiments/12c-claims-rederivation.md	Overnight WP1 (experiments/13-declared-access-order.md, results/data/declared-vs-learned-policy.csv)	Still exactly correct for layer_order_learned and for every --compute-ns-per-mib 400000 cell. Superseded at compute=0 by layer_order_declared (A-12), which reads exactly OPT (D/OPT = 1.000) at all 6 compute=0 cells in the WP1 grid — the learning cost and every suboptimal victim choice are eliminated when the workload's own timing is deterministic. The 1.07-1.78 range is now regime-dependent, not a property of "the informed policy".
Campaign 13's framing that "the application knows its access order in advance" is a claim §1 makes but the code does not implement (layer_order infers from fault-dispatch order)	experiments/11-policy-determinism.md, PROJECT_STATE §2 (the "should it be fixed" open question)	Overnight WP1 / Amendment A-12	Now implemented: layer_order_declared takes the sequence via policy_declare_sequence() at startup. layer_order_learned (the inferring variant) is retained as the comparison arm. WP1 quantified the difference (see the D/OPT row above and §2).
Overnight session-1 WP1's own findings: "layer_order_declared is Belady-optimal at compute=0 but non-deterministic and reads +2% to +47% more than the learned policy under --compute-ns-per-mib 400000"; and the §1.3 finding that declared is non-deterministic at A.2 cells 5–6	experiments/13-declared-access-order.md (session-1 body), results/data/declared-vs-learned-policy-session1.csv	Overnight session-2 WP0 fix (commit 8c15d8b) + re-sweep (results/data/declared-vs-learned-policy.csv)	The WP0 consumption-signal fix (lo_declared_dist protects seq[pos]/seq[pos-1]) reverses the compute=400000 regression: layer_order_declared now reads exactly OPT (D/OPT = 1.000) and is deterministic at all six 8 MiB cells, both compute levels. It beats layer_order_learned on bytes at 23/24 arm-D cells. Cells 5–6 become deterministic (55 / 768). Cost: one D/OPT regression at 128MiB/r=0.25/c=0 (1.06 → 1.15) and residual mild non-determinism at cell 3. Session-1 data preserved as results/data/declared-vs-learned-policy-session1.csv.
Campaign 11 Phase 2 / Campaign 13 Phase C: "arm E (prefetch) beats arm D on bytes under a real compute phase (--compute-ns-per-mib 400000), at r=0.5 and r=0.75"	experiments/08-compute-phase.md, experiments/12c-claims-rederivation.md	Overnight session-2 WP2 (experiments/14-real-model-integration.md, results/data/real-model-arms.csv)	Does not transfer to real inference. Measured per-layer compute for Qwen2.5-3B is ~51 k ns/MiB — ~1/30 of the synthetic "400000" setting where E won. On the real model E reads 7–9% more bytes than D at every ratio. E still wins latency (2× fewer demand faults, ~half the p99 inter-token latency). The synthetic finding stands as a statement about the synthetic heavy-compute regime; it is not a statement about this model.
The synthetic assumption that the model's chunk table is uniform and file_off == region_off for every chunk (every sweep through Campaign 13)	MECHANISM_SPEC.md §4 step 8, region.c build_chunk_table	Overnight session-2 WP2 Phase 2.1	A real GGUF (Qwen2.5-3B-Instruct-q4_k_m): 41 chunks, sizes 0.01–243 MiB, tensors stored in name-lexicographic not layer order, layer 21 split across two non-contiguous file chunks, final chunk's 4096-aligned end past EOF (§6.1 slack, exercised for the first time in fetch.c). budget.c / retention needed no change — already byte-based.
Session-2 WP0's "protect-current" heuristic (lo_declared_dist returns 0 for seq[pos]/seq[pos-1]) as the fix for the declared policy's non-determinism, and its costs ("~2 resident chunks", "128MiB/r=0.25/c=0 D/OPT 1.06 → 1.15", "residual mild non-determinism at cell 3")	experiments/13-declared-access-order.md (session-2 amendment), commit 8c15d8b, PROJECT_STATE §3	Final-session Phase 2 (experiments/16-consumption-signal.md, commit 8227670)	Superseded by --consumption-signal all-threads (fire pager_notify_access on full-thread completion of a chunk, not thread-0 start). Deterministic at all six A.2 cells including cell 3; 128 MiB D/OPT recovered to 1.08 / 1.04 / 1.00 (r = 0.25 / 0.5 / 0.75); reads ≤ the heuristic at every measured cell. The heuristic is now --protect-current, default off — it is redundant with the exact signal and mildly over-protective (all-threads+on reads more than all-threads+off). It remains available and unit-tested for the case where the caller's signal is inexact.
WP2's "arm D ≈ arm A at r=0.25 on the real model" and its stated cause ("the kernel keeps memory.max (782 MiB) resident, exceeding D's 526 MiB budget")	experiments/14-real-model-integration.md (verdict, expectation 2), results/data/real-model-arms.csv	Final-session Phase 1 (experiments/15-equal-budget-baseline.md, commit aff3e02)	The cause was correct but was a fixable confound, not an inherent property: WP2 ran arm A at memory.max = B + 256 MiB. At a genuinely equal budget (arm A memory.max = B), arm D beats arm A on bytes at every ratio — D/A = 0.98 / 0.89 / 0.71 / 0.58 / 0.52. r=0.25 is parity (within a disclosed ~50 MiB pager-favouring asymmetry); r ≥ 0.375 is a clear D win (−11 % to −48 %). WP2's r ≥ 0.5 wins (−24 % / −51 %) become −29 % / −48 % — larger at r=0.5, marginally smaller at r=0.75 (arm-A reclaim variance). D/OPT 1.09–1.14 (later re-measured 1.08–1.13 with the A-14 fixes — see the LIVELOCK FIX row below).
WP2's mechanism for arm E's r=0.25 collapse: "prefetch pinned-retention + the WP0 current-chunk protection leave too few evictable chunks; demand fetches spin in handle_absent's ensure_budget retry loop"	experiments/14-real-model-integration.md ("Arm E's collapse at ratio 0.25"), BLOCKERS.md (session-2 BLOCKER 2)	Final-session Phase 3 (experiments/17-prefetch-collapse.md, BLOCKERS.md FINDING 1, commit 6a47ca9)	Wrong mechanism. SIGUSR1 dump at the hang: resident_bytes 148–403 MiB of a 526 MiB budget (budget not binding), 2–3 RESIDENT+unpinned chunks available (victims exist), stat_infeasible = 0, stat_pin_broken = 0; one chunk stuck in FETCHING with every fetch worker idle in futex_do_wait and the main thread in handle_userfault. It is a latent deadlock — a demand fault deduped against an orphaned FETCHING slot — not eviction starvation. Phase 2's --protect-current off default resolves it; --prefetch-retention none resolves it with the lowest bytes. Arm E still has no advantageous config at r ≤ 0.375 (best completing read: 166.5 GB > arm D's 126.1).
Final-session Phase 3's own "it is a latent hard deadlock — an orphaned FETCHING slot" framing (the row above); and Phase 2's --protect-current off as the real-model default	experiments/17-prefetch-collapse.md, experiments/16-consumption-signal.md, PROJECT_STATE §3/§5 (final-session versions)	Cleanup session Phase 1 (experiments/17b-livelock-diagnosis.md, commits a0e7259, 214b871)	Phase 3's SIGUSR1-only dump (resident_bytes well under budget, workers in futex_do_wait) was read as "pager idle → deadlock". gdb backtraces + a 420 s counter trace show workers actively fetching the whole time and every counter climbing linearly — it is a LIVELOCK (~90× I/O amplification, stat_infeasible = 0, would finish in ~6 h), driven by protect_current on shielding the previous token's chunks (the real-model notify lags) and breaking prefetch_admit. The pager_abandon_fetch + FETCHING watchdog (A-13) is a real fix for the orphaned-slot class but not for this livelock. --protect-current off regresses arm D by +67–78 % on the real model (deterministic) — residctl_llama.c reverted to default on; replay_main.c keeps off.
The cleanup session's "livelock from two individually-correct mechanisms (retention × protection) — recorded, not fixed; mitigate with protect_current on"; and "--protect-current off regresses arm D by +67–78 % on the real model"; and residctl_llama.c default protect_current on	experiments/17b-livelock-diagnosis.md, results/claims.md Claim 10, PROJECT_STATE §3/§5 (cleanup-session versions), results/writeup-package.md §§2/3/5/6	LIVELOCK FIX session (A-14; experiments/19-livelock-fix.md, experiments/21-livelock-real-model.md, results/data/livelock-arm-e-protect-on.csv, results/data/livelock-arm-d-protect-on.csv)	Source review found the actual cause: lo_declared_dist() scanned for d = 1..seq_len so the actively-consumed chunk seq[pos] ranked as coldest (off-by-one origin), fed by a cursor that lagged a full layer because eval_cb() acted on the post-compute callback pass, with token_embd receiving zero signals (eval_cb matched "inp_embd"; the node is "embd"). All three fixed. Arm E completes at every ratio with protect_current on or off (Phase 3b: the exact prior-livelock config finishes in ~44–54 s, was rc=124/~6 h). The "+67–78 % for protect-off" was the same two bugs — with the fixes, protect_current on vs off moves arm D by ≤ 1.8 % (Phase 3c), so residctl_llama.c default flips to off (both paths off). A-13 is retained, independently, as the fix for the real orphaned-slot class.
The final-session Phase 1 real-model arm-D column (126.1 / 98.4 / 79.3 / 60.6 / 43.4 GB; D/OPT 1.09–1.14; D/A 0.98 / 0.89 / 0.71 / 0.58 / 0.52; throughput 0.91 → 1.95 t/s, 2.1×) and arm C throughput ~0.8 t/s	results/data/real-model-bytes-by-budget.csv/.md, results/figures/06-real-model-bytes.csv, results/figures/07-throughput-scaling.csv, table1_final_real_model.*, results/claims.md Claims 2/4/7/8/9, results/writeup-package.md §1	LIVELOCK FIX Phase 3 (results/data/livelock-real-model-arms.csv; --protect-current off + the four A-14 fixes; n=3, deterministic)	Re-measured: arm D 125.0 / 98.5 / 78.9 / 59.7 / 41.9 GB (within noise of the pre-fix column — the fixes barely move arm D), D/OPT 1.08–1.13 (65-pass OPT), D/A 0.97 / 0.89 / 0.71 / 0.57 / 0.50, throughput 1.12 → 2.85 t/s (2.5×, monotone — the Phase 1 r=0.5→0.625 dip did not reproduce). Arm C ~1.0 t/s (byte-identical thrashing; between-session machine-speed variance). Arm A is unchanged (no pager) and stays sourced from results/data/real-model-bytes-by-budget.csv. Figures 6/7 + Table 1 regenerated from results/data/livelock-real-model-arms.csv for C/D/E; Figures 1–5 unchanged (synthetic path byte-identical, Phase 2).