harness(rg3): RG4-K7 — kill-during-rollback-apply covered against the mock (INV-10)

K7 (SIGKILL mid rollback-apply) was previously deferred because its
never-committed scenario forked BELOW the finalized anchor — an intrinsically
invalid reorg the product must (and does) fatal on as an unknown-parent gap.
That was a scenario-design bug, not a mock or product limit: the product has a
real accepted rollback-apply path (portal-realtime.ts:1105-1117, exercised today
by K5-409) and mock-portal.mjs already carries the handleRollbackApply handler.

This authors a correct scenario whose fork point sits strictly ABOVE the anchor,
so the product enters reconcile -> {kind:'reorg'}, rolls back the losing suffix,
and re-applies the winning branch; a SIGKILL mid-apply resumes byte-identical.
No product code changes — harness only (mock + scenario + orchestrate + prose).

- scenarios/k7-rollback.json: window 101..106 on 'main' (finalized anchor <=102),
  a childDiscovery+awaitRedelivery handshake at 101 (deterministic non-vacuous
  vault+deposit rows, mirroring k5-canonical — no factory-redelivery race), then
  a rollbackApply forking at block 104 (parent 103:main, in-window above the
  anchor) extended to block 112 so the app completes. killAt block 105 lands the
  SIGKILL mid re-apply, after the rollback event and the first apply. The baseline
  digest oracle uses the same file via baseline_scenario_for_class fall-through
  (the killAt gate is released, not a path divergence — no separate canonical
  file needed, unlike K5's 409 negotiation).

- mock-portal.mjs handleRollbackApply: cursor discipline (204 a mis-cursored
  request, parity with handleStatus409) and a reorgApplied counter incremented
  where the cross-branch fork block is actually served, exposed in handler stats
  so a scenario that silently degraded to a plain append cannot self-report a
  vacuous pass.

- orchestrate.sh: strengthen k7RollbackNonVacuousOk to mirror k6CutoverOrganicOk
  — assert killsWithLiveStream===killCount, the empirical kill-block set is
  exactly {105}, and reorgApplied>0 per run. Candor tightened to defer ONLY
  live-protocol fork-choice fidelity to RG4/RG5; the crash-timing half is now
  mock-covered.

- VALIDATION.md 5.11 / class table: K7 reclassified from deferred to
  mock-covered (crash-timing), mapped to existing INV-10 (no new invariant);
  live-protocol fork fidelity remains RG4/RG5.

- mock-portal.test.mjs: cursorMatchesStep routes a rollbackApply match only on
  the natural post-window resume cursor.

Empirical proof (CHAOS_CLASSES=K7 CHAOS_KILLS_PER_CLASS=1, graft built fresh from
this tree): scenario picked up (not the deferral branch); baseline reorged at
block 103 and re-applied the rollback branch cleanly (no "Cannot reconcile
safely"); kill landed at block 105 mid-apply; resume byte-identical
(store+app digest identical, vault=1 deposit=1 non-vacuous, zero duplicate
finalized rows); k7RollbackNonVacuousOk=true with killBlocks=[105],
killsWithLiveStream=1, reorgApplied=1. mock-portal.test.mjs 10/10; biome clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: dzhelezov

VALIDATION.md

@@ -1722,8 +1722,14 @@ cutover-handler entry (a 409 variant + a wrong-fork variant) · K6 at the histor
 Per-class mock fidelity is recorded candidly in the harness and reproduced here — HIGH for K1/K2/K4
 (direct 200/reorg/defer flows), MEDIUM for K3/K5 (redelivery / fork-handler reachability), and
 **MEDIUM for K6** after Phase B (an organic mock cutover at the client's natural chunk-boundary
-request). K7 is **not included** in RG3 Phase B: the pre-flight hit the product unknown-parent fatal
-before a kill, so kill-during-rollback remains RG4/RG5.
+request). K7 (kill-during-rollback-apply) is added against the mock: the pre-flight fatal was a
+scenario-design bug (a fork point *below* the finalized anchor, which correctly fatals as an unknown-
+parent/deep-reorg gap), not a mock or product limit. With the fork point corrected to sit strictly
+*above* the anchor, the same product code path (`reconcile → {kind:'reorg'}` then rollback-apply,
+`portal-realtime.ts:1105`) that K5-409 already exercises drives an accepted rollback, and the SIGKILL
+lands mid-apply — so the **crash-timing** half of K7 is now mock-covered. What remains RG4/RG5 is only
+the **live-protocol fork-choice fidelity** (a real Portal 409/reorg with competing finalized sources),
+not kill-during-rollback as a class.
 
 **Result (Phase A, from the operator's evidence archive — verbatim).**
 
@@ -1749,7 +1755,7 @@ All three RG3 numeric thresholds are met: **238 ≥ 200** kills across **7 ≥ 6
 | Duplicate FINALIZED rows | **0** |
 | K6 organic cutover | **10 / 10** (every kill on block 106, the chunk boundary) |
 | K5 wrong-fork consumed in-window | **deferred** — default clean run consumed **0 / 1**; a stretched post-injection window makes the product correctly raise the wrong-fork finalize fatal |
-| K7 kill-during-rollback | **deferred** — pre-flight baseline fataled on unknown-parent/deep-reorg before any kill |
+| K7 kill-during-rollback-apply | **mock-covered (crash-timing)** — fork point corrected to sit ABOVE the finalized anchor (≤102), so `reconcile → {kind:'reorg'}` rolls back 104..106:main and re-applies the rollback branch; the SIGKILL lands mid-apply at block 105 and resumes byte-identical. `k7RollbackNonVacuousOk` self-certs kill-block=={105}, live /stream before every kill, and the mock served the cross-branch fork (`reorgApplied>0`). **Live-protocol fork fidelity remains RG4/RG5.** |
 
 The full K1–K6 moderate rerun was attempted with
 `CHAOS_KILLS_PER_CLASS=10 CHAOS_MIN_PER_CLASS=10 CHAOS_TOTAL_KILLS=70`, but it stopped before Phase-B
@@ -1793,10 +1799,24 @@ hidden:
   `fromBlock=106` with the canonical parent hash). The cutover is **organic within the mock's
   deterministic capability** — it is still not a live Portal cutover with competing finalized sources;
   that remains RG4/RG5.
-- **K7 kill-during-rollback is deferred.** The pre-flight scenario attempted to stream block 104 on a
-  rollback branch with parent block 103 while the finalized anchor was 105. The client treated that as
-  the existing unknown-parent/deep-reorg fatal (`Cannot reconcile safely`) before any kill could be
-  taken. The invariant was not weakened; K7 remains RG4/RG5.
+- **K7 kill-during-rollback-apply — crash-timing is mock-covered; live fork fidelity remains RG4/RG5.**
+  The pre-flight scenario that fataled attempted to stream block 104 on a rollback branch with parent
+  block 103 while the finalized anchor was 105 — a fork point *below* finality, which correctly fatals
+  (`Cannot reconcile safely`; a fork below the finalized floor has no safe rollback). That was a
+  **scenario-design bug**, not a mock or product limit: the mock already carries the
+  `handleRollbackApply` handler and the product carries a real accepted rollback-apply path
+  (`portal-realtime.ts:1105-1117`), exercised today by K5-409. K7 now moves the fork point strictly
+  *above* the anchor (window 101..106 on `main`, anchor ≤102, reorg forking at block 104), so
+  `reconcile` returns `{kind:'reorg'}` (not `gap`), the losing 104..106:main suffix is rolled back and
+  the `rollback` branch re-applied. The SIGKILL fires at block 105 — after the rollback event and the
+  first apply, mid re-apply — and the resume is byte-identical. `k7RollbackNonVacuousOk` self-certifies
+  non-vacuity exactly as `k6CutoverOrganicOk` does: the empirical kill-block set is `{105}`, every kill
+  followed ≥1 live `/stream` request, and the mock-side `reorgApplied` counter (>0 per run) proves the
+  cross-branch fork was actually served rather than degrading to a plain append. This is crash-timing
+  coverage of **INV-10**'s rollback arm — no new invariant. What genuinely **remains RG4/RG5** is only
+  the *live-protocol fork-choice fidelity*: a real Portal 409/reorg with competing finalized sources
+  and real parentHash chains. The mock proves the rollback-apply *code path* is crash-timing safe; it
+  does not prove the Portal emits such branches — that is RG4/RG5.
 - **The harness `invariant()` fast-path is a fatal-string tripwire, not the primary death detector.**
   It grep-matches all eight real fatal throws in `portal/portal-realtime.ts` (the gap fatal, the
   below-floor 409 fatal, the 409 no-progress cap, the deterministic-4xx fatal, the delivery-progress

harness/chaos/realtime/mock-portal.mjs

@@ -501,6 +501,7 @@ function createRuntime(initialScenario, options = {}) {
     wrongForkFinalizes: 0,
     wrongForkFinalizeConsumed: 0,
     wrongForkFinalizeRejected: 0,
+    reorgApplied: 0,
     requestLog: [],
   };
   const phaseLog = options.phaseLog;
@@ -573,6 +574,7 @@ function createRuntime(initialScenario, options = {}) {
     stats.wrongForkFinalizes = 0;
     stats.wrongForkFinalizeConsumed = 0;
     stats.wrongForkFinalizeRejected = 0;
+    stats.reorgApplied = 0;
     stats.requestLog = [];
     pendingWrongForkHead = undefined;
     skipWrongForkRejection = false;
@@ -893,13 +895,33 @@ function createRuntime(initialScenario, options = {}) {
   };
 
   const handleRollbackApply = async (step, body, res) => {
+    // Cursor discipline (parity with handleStatus409): only serve the reorg branch on the client's
+    // natural resume cursor. `stream()` already 204s a mis-cursored request via isCursorStep, but a
+    // defensive re-check here makes a stray request 204 rather than mis-deliver the rollback branch.
+    if (step.match !== undefined && cursorMatchesStep(step, body) === false) {
+      stats.r204 += 1;
+      res.writeHead(204);
+      res.end();
+
+      return;
+    }
+
     const reorgBlock = Number(step.reorgBlock ?? streamCursor - 1);
     const count = Math.max(0, Number(step.count ?? 1));
     const branchId = step.branch ?? step.branchId ?? 'rollback';
     const parentBranchId = step.parentBranch ?? 'main';
     res.writeHead(200, { 'content-type': 'application/x-ndjson' });
     for (let i = 0; i < count; i++) {
       const number = reorgBlock + i;
+      // Count the reorg branch actually being served: the fork block (i === 0) carries a CROSS-branch
+      // parentHash (parent on `parentBranchId`, self on `branchId`), which is what drives the product's
+      // reconcile -> {kind:'reorg'} rollback-apply. A scenario that silently degraded to a plain
+      // tip-append (fork point == branch) has parentBranchId === branchId and never increments this,
+      // so a vacuous "append masquerading as rollback" cannot self-report a K7 pass.
+      if (i === 0 && parentBranchId !== branchId) {
+        stats.reorgApplied += 1;
+      }
+
       const phase = gatePhaseForBlock(step, number);
       if (phase !== undefined) {
         const open = await gate(

harness/chaos/realtime/mock-portal.test.mjs

@@ -227,3 +227,40 @@ test('cursorMatchesStep: routes redelivery and 409 steps by request cursor', ()
     false,
   );
 });
+
+test('cursorMatchesStep: rollbackApply fires only on the natural post-window resume cursor', () => {
+  // K7: after streaming 101..106 on `main`, the client's natural next request is
+  // fromBlock=107 with parentBlockHash = hashBlock(106, 'main'). The rollbackApply step's
+  // `match` must fire there — and NOT on the earlier in-window cursors — so the reorg branch
+  // is served on the resume request rather than mis-delivered mid-stream.
+  const step = {
+    type: 'rollbackApply',
+    reorgBlock: 104,
+    count: 3,
+    branch: 'rollback',
+    parentBranch: 'main',
+    match: { fromBlock: 107, parentBlock: 106, parentBranch: 'main' },
+  };
+
+  assert.equal(
+    cursorMatchesStep(step, {
+      fromBlock: 107,
+      parentBlockHash: hashBlock(106, 'main'),
+    }),
+    true,
+  );
+  assert.equal(
+    cursorMatchesStep(step, {
+      fromBlock: 106,
+      parentBlockHash: hashBlock(105, 'main'),
+    }),
+    false,
+  );
+  assert.equal(
+    cursorMatchesStep(step, {
+      fromBlock: 107,
+      parentBlockHash: hashBlock(106, 'rollback'),
+    }),
+    false,
+  );
+});

harness/chaos/realtime/orchestrate.sh

@@ -511,7 +511,7 @@ const candor = {
   K4: 'idle / 204 retry path',
   K5: "synthetic fork via stale-parentBlockHash 409 (client consumes the canonical replacement chain and resumes byte-identically) PLUS a wrong-fork-finalize handler-REACHABILITY probe. handlerStats.wrongForkFinalizes counts mock-side handler entries (reachability) ONLY — the counter increments on handler entry, before the gate, so on a killed run the splice may not even execute; the injected wrong finalized head is NOT consumed by the client here (finalize poll cadence ~4s > the post-injection scenario window ~1.8s), so this is not clean-resume wrong-fork evidence. The client-side wrong-fork-finalize FATAL guard (hash-mismatch at finality) is proven deterministically by portal-realtime.test.ts ('a finalize whose canonical hash mismatches the local block is FATAL', finalizePollMs:0). Real L1-reorg wire fidelity NOT exercised.",
   K6: "kill lands at the backfill→live chunk boundary (cutoverGate fires on the client's natural fromBlock=106 request, not a fixed block), so resume re-drives an organic cutover within the mock's deterministic capability. Probe-retry wire fidelity (the 3-attempt loop in clampFinalizedToPortalHead) NOT exercised — mock's /finalized-head succeeds first try; probe-retry fidelity rests on portal-realtime-wire.test.ts.",
-  K7: 'K7: kill lands mid rollback-apply (client sees parent mismatch at block 104 and must roll back to 103 before applying the rollback branch). Mock-driven: the rollback is scripted, not a live L1 reorg. Live reorg-during-rollback fidelity is RG4/RG5.',
+  K7: "kill lands mid rollback-apply: after 101..106 on main, the client resumes at fromBlock=107/parent=106:main, the mock serves a reorg branch forking at block 104 (104:rollback carries parent 103:main, in-window above the <=102 finalized anchor), so reconcile -> {kind:'reorg'} rolls back 104..106:main and re-applies the rollback branch 104..112 (extended to PONDER_END so the app completes); the SIGKILL fires at block 105, AFTER the rollback event and the first apply, mid re-apply. The fork point is strictly ABOVE the finalized anchor (<=102) so this is an ACCEPTED reorg, never the below-floor gap fatal. reorgApplied counts the mock actually serving the cross-branch fork block, so a scenario that degraded to a plain append cannot self-report a K7 pass. Mock-driven crash-timing coverage of INV-10's rollback arm (same product code path from portal-realtime.ts:1105 as K5-409's accepted reorg). What is NOT proven here: live-protocol fork-choice fidelity — a real Portal 409/reorg with competing finalized sources and real parentHash chains — remains RG4/RG5.",
 };
 const lines = fs.existsSync(process.env.RECORDS)
   ? fs.readFileSync(process.env.RECORDS, 'utf8').trim().split(/\n+/).filter(Boolean)
@@ -534,6 +534,7 @@ for (const klass of classes) {
     acc.wrongForkFinalizes += Number(s.wrongForkFinalizes ?? 0);
     acc.wrongForkFinalizeConsumed += Number(s.wrongForkFinalizeConsumed ?? 0);
     acc.wrongForkFinalizeRejected += Number(s.wrongForkFinalizeRejected ?? 0);
+    acc.reorgApplied += Number(s.reorgApplied ?? 0);
     return acc;
   }, {
     r204: 0,
@@ -544,6 +545,7 @@ for (const klass of classes) {
     wrongForkFinalizes: 0,
     wrongForkFinalizeConsumed: 0,
     wrongForkFinalizeRejected: 0,
+    reorgApplied: 0,
   });
   // Distinct empirical kill blocks (from each run's kill.phase.log) — proves K2's varied mid-stream
   // coverage rather than one repeated fixed block. `killBlocksRecorded` keeps every row's non-null
@@ -556,6 +558,11 @@ for (const klass of classes) {
   const killsWithLiveStream = rows.filter((r) => Number(r.killStats?.stream ?? 0) > 0).length;
   const wrongForkRows = ok.filter((r) => r.variant === 'wrongfork');
   const wrongForkConsumedRuns = wrongForkRows.filter((r) => Number(r.stats?.wrongForkFinalizeConsumed ?? 0) > 0).length;
+  // K7 non-vacuity: count ok runs whose FINAL mock stats recorded the reorg branch actually served
+  // (reorgApplied > 0) on the resumed run. A scenario that silently degraded to a plain append (fork
+  // point drifting to the tip) never increments the mock-side counter, so a vacuous K7 pass cannot
+  // self-report reorgAppliedRuns === killCount.
+  const reorgAppliedRuns = ok.filter((r) => Number(r.stats?.reorgApplied ?? 0) > 0).length;
   perClass[klass] = {
     scenarios: [...new Set(rows.map((r) => r.scenario).filter(Boolean))],
     variantLabels: [...new Set(rows.map((r) => r.variant).filter(Boolean))].join(', '),
@@ -565,6 +572,7 @@ for (const klass of classes) {
     killsWithLiveStream,
     wrongForkRunCount: wrongForkRows.length,
     wrongForkConsumedRuns,
+    reorgAppliedRuns,
     cleanResumeCount: ok.length,
     digestMatchCount: digestMatch.length,
     duplicateFinalizedRows: dupCount,
@@ -615,12 +623,23 @@ const acceptance = {
     && perClass.K6.killsWithLiveStream === perClass.K6.killCount),
   k2KillSpreadOk: !hasClass('K2') || perClass.K2.killBlocks.length >= 3,
   // Left `undefined` (not `true`) when K7 is absent so the guards below skip it entirely — a class
-  // that never ran must not self-report a vacuous "non-vacuous: true" pass in RESULT.md.
+  // that never ran must not self-report a vacuous "non-vacuous: true" pass in RESULT.md. When present,
+  // mirror k6CutoverOrganicOk's non-vacuity self-cert so a scenario that silently degraded to a plain
+  // append (fork point drifting to the tip, so the kill never lands mid-rollback-apply) cannot report
+  // a byte-identical pass: every K7 kill must have seen a live /stream request before the kill
+  // (killsWithLiveStream), the empirical kill-block set must be exactly {105} (the mid-apply block),
+  // and every ok run's FINAL mock stats must record the reorg branch actually served
+  // (reorgAppliedRuns === killCount) — the mock-side reachability floor for the load-bearing splice.
   k7RollbackNonVacuousOk: !hasClass('K7')
     ? undefined
     : (perClass.K7.killCount > 0
       && perClass.K7.cleanResumeCount === perClass.K7.killCount
-      && perClass.K7.digestMatchCount === perClass.K7.killCount),
+      && perClass.K7.digestMatchCount === perClass.K7.killCount
+      && perClass.K7.killsWithLiveStream === perClass.K7.killCount
+      && perClass.K7.killBlocksRecordedCount === perClass.K7.killCount
+      && perClass.K7.killBlocks.length === 1
+      && perClass.K7.killBlocks[0] === 105
+      && perClass.K7.reorgAppliedRuns === perClass.K7.killCount),
 };
 acceptance.accepted = acceptance.totalKillsOk
   && acceptance.perClassKillsOk
@@ -679,6 +698,7 @@ for (const klass of classes) {
   md.push(`- kill blocks (distinct, from kill.phase.log): ${c.killBlocks.length > 0 ? c.killBlocks.join(', ') : 'n/a'}`);
   md.push(`- kills after >=1 live /stream request: ${c.killsWithLiveStream} / ${c.killCount}`);
   if (klass === 'K5') md.push(`- wrong-fork consumed runs: ${c.wrongForkConsumedRuns} / ${c.wrongForkRunCount}`);
+  if (klass === 'K7') md.push(`- reorg-applied runs (mock served the rollback branch): ${c.reorgAppliedRuns} / ${c.killCount}`);
   md.push(`- handler stats: ${JSON.stringify(c.handlerStats)}`);
   md.push(`- fidelity: ${c.fidelity} — ${c.candor}`);
   if (c.failures.length > 0) md.push(`- failures: ${JSON.stringify(c.failures)}`);

harness/chaos/realtime/scenarios/k7-rollback.json

@@ -0,0 +1,63 @@
+{
+  "chainId": 1,
+  "genesis": {
+    "number": 100,
+    "timestamp": 1700001200
+  },
+  "finalizedHeadSeq": [
+    { "number": 100 },
+    { "number": 100 },
+    { "number": 102 },
+    { "number": 102 }
+  ],
+  "steps": [
+    {
+      "type": "childDiscovery",
+      "block": 101,
+      "child": "0x1111111111111111111111111111111111111111",
+      "transactionHash": "0xaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+      "logIndex": 0
+    },
+    {
+      "type": "awaitRedelivery",
+      "block": 101,
+      "logs": [
+        {
+          "block": 101,
+          "event": "ProxyCreated",
+          "proxy": "0x1111111111111111111111111111111111111111",
+          "transactionHash": "0xaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+          "transactionIndex": 0,
+          "logIndex": 0
+        },
+        {
+          "block": 101,
+          "event": "Deposit",
+          "vault": "0x1111111111111111111111111111111111111111",
+          "sender": "0x2222222222222222222222222222222222222222",
+          "owner": "0x3333333333333333333333333333333333333333",
+          "assets": "123456789",
+          "shares": "987654321",
+          "transactionHash": "0xbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+          "transactionIndex": 0,
+          "logIndex": 1
+        }
+      ]
+    },
+    {
+      "type": "blocks",
+      "count": 5,
+      "branch": "main",
+      "turnDelayMs": 10
+    },
+    {
+      "type": "rollbackApply",
+      "reorgBlock": 104,
+      "count": 9,
+      "branch": "rollback",
+      "parentBranch": "main",
+      "match": { "fromBlock": 107, "parentBlock": 106, "parentBranch": "main" },
+      "killAt": { "block": 105, "phase": "K7-rollback" }
+    }
+  ]
+}