|
3 | 3 | The `portal/` layer is organised around **explicit, provable invariants**. Every module has a functional |
4 | 4 | core (pure, unit-tested) behind an imperative shell; the core asserts these invariants at runtime |
5 | 5 | (`invariant("INV-…", …)` from `portal-invariant.ts`) and the test suite proves them with property-based |
6 | | -tests (`fast-check`, seed-pinned for deterministic CI). IDs are stable (INV-1…INV-20) and grep-able |
| 6 | +tests (`fast-check`, seed-pinned for deterministic CI). IDs are stable (INV-1…INV-21) and grep-able |
7 | 7 | across **doc ⟷ code ⟷ test**. (INV-19 is the low-edge chunk-coverage guard from #50; INV-20 is the |
8 | | -ancestor-error trace cascade — the numbering is chronological, not table order.) |
| 8 | +ancestor-error trace cascade; INV-21 is the root-frame gas parity — the numbering is chronological, not |
| 9 | +table order.) |
9 | 10 |
|
10 | 11 | **Runtime modes** (`PORTAL_CHECKS`, default `on`): `off` disables all checks; `on` runs the O(1) checks |
11 | 12 | and throws `InvariantViolation` (a loud crash beats silent corruption); `strict` additionally runs the |
@@ -37,6 +38,7 @@ Legacy `C#` tags from the pre-refactor comments are mapped where they existed. |
37 | 38 | | **INV-17** | **Factory-children write-side idempotence.** Re-flushing an already-persisted child set does NOT durably duplicate rows: before every `insertChildAddresses`, the flush is deduped against the store's persisted rows (`getChildAddresses`) and a child is inserted only if it is not yet persisted OR is re-discovered at a STRICTLY LOWER creation block — the write-side analogue of the read-side min-merge (LEAST semantics). So a resumed or re-running writer that re-queues the same children from a fresh in-memory discovery queue inserts nothing; a lower re-discovery updates the effective min; equal/higher re-flushes are no-ops. The flush is FIRST grouped by STORE IDENTITY (factory minus `id`/`sourceId`, matching how the sync-store keys `factory_addresses`) so two config sources aliasing the same on-chain factory (identical factory minus `id`/`sourceId`) collapse to ONE read + ONE dedupe + ONE insert — otherwise both would read absence and each insert the same children (a first-flush duplicate the read-side min-merge cannot repair). The persisted lookup is LOWERCASE-normalized (children are lowercased at discovery) so a checksummed pre-existing row is still deduped. Addresses #53 (the fork-side hardening it proposes). **Known cost (candor):** the guard is a live UNCACHED `getChildAddresses` (SELECT of all rows) per canonical pending factory per flush-bearing interval — over a long accreting backfill this is worst-case ~O(N²) in the child count. It is left uncached in this PR (correctness first); a per-process read-through cache of the persisted min-map is the documented follow-up candidate. | Upstream's `insertChildAddresses` is a plain `INSERT` with no `ON CONFLICT`, and `ponder_sync.factory_addresses` has no `UNIQUE (factory_id, chain_id, address)` (the only sync-store table with no uniqueness story) — so a second writer on the same store durably duplicates the whole child set (the ×2-children observation came from the chaos DRIVER running two concurrent live instances — a fixed harness-driver defect — NOT from SIGKILL/resume itself, which is single-writer; see SCOPE below). App behavior is unaffected (the only consumer, `getChildAddresses`, min-merges to a set), but store-identity/digest tooling breaks and the table grows unbounded under repeated violations. The in-memory min-merge guard (INV-15) suppresses re-queueing WITHIN one process; a second live process has a fresh queue and needs a store-side guard. **SCOPE (candor):** the read→dedupe→insert→`insertIntervals` sequence all runs inside ONE store transaction per interval (the `syncStore` is tx-scoped — created from the tx handle in `runtime/historical.ts:1310-1352`), so for a single writer the guard is TRANSACTIONAL, not merely sequential: no interleaving read/insert can slip a duplicate past it, and a failed flush rolls the whole interval back. This fully closes the resumed single-writer case (the realistic production shape — production holds a single-writer app lock). The residual risk is genuinely ONLY the two-transactions / two-concurrent-processes case: two writers in two transactions can both read absence and both `INSERT`. That TOCTOU race the portal-layer dedupe does NOT close; only a DB `UNIQUE` + `ON CONFLICT DO UPDATE … LEAST` would, and the fork cannot add that cleanly — `factory_addresses` is created by a hash-keyed kysely migration (`2025_02_26_0_factories`), so a real constraint needs a NEW duplicate-collapsing migration in upstream `migrations.ts` for both pinned versions, materially larger blast radius. The #53 concurrent writer was a fixed harness-driver defect; the residual concurrent-writer gap is documented and left to a future DB-constraint change. | `portal.ts` reads `syncStore.getChildAddresses({ factory })` per pending factory before the flush and keeps a child only when `persisted.get(address) === undefined \|\| persisted.get(address) > block`; the store method is upstream, so the guard lives at the fork's own call site. | — (idempotence property; the duplicate it prevents has no safe runtime signal on the app path, so it is proven by test). | `portal.test.ts` — "regression (#53, write-side idempotence): a SECOND writer re-flushing an already-persisted child set does NOT re-insert it — the store keeps exactly one row at the min block" (mutation-verified: reverting the dedupe to a plain flush fails it — `expected […] to have a length of 1 but got 2`), "regression (#53, write-side idempotence): a re-discovery at a STRICTLY LOWER creation block is re-inserted so read-side LEAST resolves to the min (equal/higher blocks stay deduped)" (mutation-verified: a blanket `DO NOTHING` predicate `prev !== undefined` fails it — `expected [] to have a length of 1 but got +0`), "regression (#53, alias-hole): two sources whose factories differ ONLY in id/sourceId (store aliases) discovering the same child in one interval end with EXACTLY ONE store row — not two" (mutation-verified: reverting the canonicalization to key by `factory.id` fails it — the store ends with the child ×2), "regression (#53, case-normalization): a CHECKSUMMED pre-existing store row is deduped against the lowercased discovered child" (mutation-verified: dropping the lowercase-normalize of the persisted map fails it — the checksummed row no longer matches so the child is re-inserted). | |
38 | 39 | | **INV-18** | **Bounded-cutover skip.** An end-capped chain (configured end ≤ finalized) is never live-probed at the historical→realtime cutover, and a fully-bounded app does zero cutover refetches — with omnichain refinements (pending-aware break, two-phase probe, clamp bypass) that never drop parked cross-chain events. **[INV-18 in detail ↓](#inv-18-in-detail)** | — | — | — | — | |
39 | 40 | | **INV-20** | **Ancestor-error trace cascade.** Portal historical trace assembly replicates stock-ponder's ancestor-error cascade: within one transaction's trace tree a reverted ancestor's `error`/`revertReason` smears onto every descendant that lacks its OWN error, transitively (an inheriting frame becomes a source for its own children); a frame with its own error keeps it (own-error-wins). So `traces.error`/`traces.revert_reason` are byte-identical to the RPC store path. The inherited `revertReason` is assigned UNCONDITIONALLY (even when the ancestor's is `undefined`) — byte-exact with upstream `rpc/actions.ts`. | Portal delivers geth-faithful per-frame errors and never invokes upstream's `debug_traceBlock*` DFS, so a succeeded child of a reverted parent stored `error=null` where the RPC path inherits the parent's error — a `traces.error`/`revert_reason` divergence between the two store paths. Realtime/fallback already flow through the untouched RPC cascade, so the fork DB must be uniformly ponder-shaped. | `portal-assemble.ts`: `cascadeTraceErrors(sorted)` derives each frame's effective error over the DFS-sorted (`cmpTraceAddr`) frames of a tx, keyed by traceAddress prefix (parent of `[a,b,c]` is `[a,b]`); `rankTraces` applies it to each produced frame. Applied ONLY on the historical path — the realtime path rides the untouched upstream cascade and cannot double-apply. Non-prefix-closed input inherits nothing (conservative; never fabricates a smear; cannot occur for geth-faithful trees). | — (byte-identity property; the divergence it prevents has no safe runtime signal on the app path, so it is proven by test against the RPC-store oracle). | `portal-assemble.test.ts` — "INV-20: real reverted tx — the whole reverted subtree inherits the ancestor error (oracle: RPC store)" (real eth block 20351061 tx 26 fixture), "INV-20 (a): a child with its OWN distinct error SURVIVES (own-error branch wins)", "INV-20 (b): ≥3-deep nesting inherits the ancestor error transitively", "INV-20 (c): a non-reverted tx keeps every frame error/revertReason null", "INV-20 (d): an inheriting child of an ancestor with an error but NO revertReason gets error set AND revertReason === undefined (unconditional assignment)". Mutation-verified: neutering the cascade (empty effective-error map / skipping the applied `cascaded`) fails INV-20 / (a) / (b) / (d) while (c) still passes. Upstream divergence documented in VALIDATION.md §5. | |
| 41 | +| **INV-21** | **Root-frame gas parity.** Portal historical trace assembly stores the transaction's FULL gasLimit on the ROOT trace frame (`traceAddress []`, the top-level call), byte-identical to the stock-ponder RPC store. Portal serves Parity-style traces whose root `action.gas` is gasLimit MINUS the EIP-2028 intrinsic (base 21000 + 16/nonzero + 4/zero calldata byte + access-list + creation costs), while geth callTracer — which the RPC path stores — puts full tx.gasLimit on the top frame UNCONDITIONALLY. Non-root frames' gas already matches both paths and is left untouched. | Portal delivers Parity `action.gas` (intrinsic-reduced) on the root frame where geth's callTracer top frame carries the full gasLimit, so the Portal store wrote `traces.gas` at the root that differed from the RPC store by exactly the tx's intrinsic (gas_used byte-identical) — a `traces.gas` divergence between the two store paths. Realtime/fallback ride the untouched RPC path, so a mixed backfill+realtime fork DB would carry a path-dependent root-frame gas with an invisible seam at the cutover — the fork DB must be uniformly ponder-shaped. | `portal-assemble.ts`: `buildTraces` overrides the root frame's gas with the parent tx's gasLimit (`frame.gas = hx(rawTx.gas)`) when `traceAddress.length === 0` and the parent tx is present. Keyed on `traceAddress`, NOT the DFS rank (a chunk can hold only deep frames). No intrinsic math — geth's top-frame gas equals the gasLimit exactly, so the override is the raw gasLimit. Applied at the store-write site only; the shared trace matcher (`buildRawTraceMatcher`/`parityToCallFrame`) is untouched (trace matching never reads gas). | — (byte-identity property; the divergence it prevents has no safe runtime signal on the app path, so it is proven by test against the RPC-store oracle). | `portal-assemble.test.ts` — "INV-21: real diff-root tx — the root frame carries the tx gasLimit (oracle: RPC store), non-root untouched" (real eth block 19068568 tx 7 Uniswap V2 swap: pre-fix Portal `action.gas` 0x74700 = gasLimit − intrinsic vs RPC/gasLimit 0x7a120; non-root child 0x3616b already byte-identical), "INV-21 (a): an already-matching root … is unchanged — idempotent", "INV-21 (b): a rootless trace chunk (only a deep frame) is NOT rewritten". Mutation-verified: reverting the root override (guard never fires ⇒ Parity `action.gas` at root) fails INV-21 (`expected '0x74700' to be '0x7a120'`); (a) and (b) still pass. Grounded on a real captured Portal-vs-RPC store diff (eth mainnet): 133/133 divergences gas-only, at the root, gas_used byte-identical, RPC.gas == tx gasLimit on every root frame. Upstream divergence documented in VALIDATION.md §5. | |
40 | 42 |
|
41 | 43 | ## Invariants in detail |
42 | 44 |
|
|
0 commit comments