Skip to content

Commit aa9761f

Browse files
committed
Decouple live ingestion from wallet accounting and repair V4 owners
1 parent f482812 commit aa9761f

13 files changed

Lines changed: 2438 additions & 404 deletions

docs/PUBLIC_API.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,13 @@ responses distinguish complete collected-fee totals from partial evidence:
263263
`episode_selection: "last_activity_within_window"` and
264264
`financial_scope: "lifetime_of_selected_episodes"`. Do not treat these
265265
values as cash flows earned exclusively during the requested window.
266+
- Owner-list `financials.through_order` is the canonical
267+
`[block_number, transaction_index, log_index]` boundary captured in the same
268+
WAL snapshot as the financial aggregates. `financials.as_of` describes that
269+
boundary, not the time of the latest observed activity. Pending projection
270+
work or later activity keeps `financials.pending` true without advancing the
271+
captured boundary. A completed snapshot is distinct from complete history,
272+
trace attribution, pricing, and fee evidence.
266273

267274
Fresh canonical activity does not imply complete transaction enrichment or
268275
historical accounting. Preserve both freshness and completeness qualifications.

docs/RPC_OPERATIONS.md

Lines changed: 37 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,12 @@ ABI revert data. The pinned-state decoder uses that evidence to recognize
3030
Keeping only the provider message loses this proof and retries valid receipts
3131
indefinitely. Unproven absence and unrelated errors still fail enrichment.
3232

33+
Pinned-state batches retain successful siblings when one contract call reverts;
34+
they do not repeat the batch serially. A verified same-receipt mint proves the
35+
parent-block position absent, but the current-block position read remains
36+
required. V3 receipts do not wait for trace capability. Successful receipts
37+
containing V4 PoolManager liquidity modifications still require `callTracer`.
38+
3339
## Goldsky measurements and limits
3440

3541
A small anonymous-output probe from the production host verified the donated provider privately: chain 4663; exact matching block/header and log digests against the local node; old block headers; USDG `decimals()` at blocks 30,000,000 and 56,400,000; receipts; `debug_traceTransaction` with `callTracer`; and a four-item JSON-RPC batch. The local pruned node could not answer those archive-state calls. Individual successful Goldsky requests in this probe took roughly 76–352 ms; these are samples, not percentile/SLA claims.
@@ -51,12 +57,18 @@ Separate durable cursor progress from the live observed feed. A moving live tape
5157

5258
Production sampling found wallet aggregation repeatedly restarting whenever ingestion advanced its revision. This could keep LP Wallets at `SYNCING` indefinitely while consuming CPU needed by the indexer. Wallet totals, gas and coverage now come from one completed WAL read snapshot; appends trigger a later refresh instead of recursive recomputation. Canonical-branch changes still invalidate service publications. The regression exercises an indexer append during an actual wallet read.
5359

54-
Late historical events rebuild only the affected position's changed accounting rows rather than deleting and rewriting its entire projection. This preserves synchronous, atomic financial updates while reducing write amplification. Existing durable metadata counters are reused on restart; full-table counts initialize missing counters only.
60+
The running service commits raw events, cursors, balance jobs, and coalesced
61+
accounting jobs atomically. A separate accounting worker replays each affected
62+
position from an immutable WAL snapshot without holding the ingestion writer.
63+
Its short publication transaction writes only changed accounting rows. A newer
64+
same-branch generation retains the job after publishing the coherent snapshot;
65+
a changed canonical epoch rejects it. Existing financial rows remain readable
66+
while the queue drains, but cannot be represented as current accounting.
5567

5668
Header, event, and search writes use parameterized multi-row inserts bounded by
5769
SQLite's variable limit. This avoids handing the Python interpreter to competing
58-
valuation workers between every inserted row. The outer durable transaction and
59-
its atomic accounting/cursor boundary are unchanged.
70+
valuation workers between every inserted row. The outer durable transaction
71+
preserves the raw event/cursor/job boundary.
6072

6173
The CLI caps Python's thread-switch interval at 1 ms before starting workers,
6274
preserving an already-shorter interval. This reduces interpreter handoff delays
@@ -116,14 +128,26 @@ balance, and repricing batches reserve historical work while preferring recent
116128
ready records; a delayed retry no longer stops unrelated repricing. Identity
117129
replay likewise alternates recent and historical work.
118130

119-
Metadata RPC fetches use the bounded enrichment workers and commit one batch.
131+
Schema version 8 adds a generation to receipt jobs. Completion checks the
132+
canonical block hash and generation inside the same transaction as enrichment;
133+
an older in-flight receipt cannot erase newer identity or retry work.
134+
The accounting queue and its bootstrap checkpoint survive restart without
135+
discarding published positions, episodes, coverage, or cursors.
136+
137+
Receipt enrichment, repricing, and accounting continue during live catch-up.
138+
Receipt workers refill independently rather than waiting for the slowest member
139+
of a batch. Metadata, identity replay, and bounded source repairs run separately
140+
from receipt scheduling. Follow `pending_accounting` as well as enrichment and
141+
repricing queues; a small block gap does not prove those queues are complete.
142+
143+
Metadata RPC fetches remain bounded and commit one batch.
120144
V3 balances use a batched pinned-state request and atomic snapshot commit.
121145
Identity replay commits canonical events and queue completion together.
122146
Price sample, reserve, mark, and state inserts use the same multi-row helper
123147
as event ingestion.
124148

125149
Existing V3 NFT positions whose initial add precedes a verified mint in the
126-
same transaction are repaired by the normal projection lane. A default pass
150+
same transaction are repaired by the separate source-repair lane. A default pass
127151
examines at most 8,192 indexed add events newest-first, selects only affected
128152
positions within reviewed V3-manager ranges, and reprojects at most 32 positions.
129153
The event cursor stops at the last selected repair when the write limit is
@@ -134,6 +158,14 @@ the accounting schema version is unchanged, so startup does not replay the
134158
entire ledger. A read-only production probe admitted 16 repairs from 8,192
135159
events in 0.092 s.
136160

161+
V4 source repair correlates already-traced manager positions with canonical NFT
162+
transfers from the same receipt, including receipts without an auxiliary
163+
PositionManager modification event. It preserves the owner at each action's
164+
log order and retains the stored trace, state, cash-flow, and fee evidence.
165+
The bounded `v4_owner_correlation_repair_v1` checkpoint resumes after restart.
166+
Fees before a transfer stay with the prior owner; receiving the NFT does not
167+
prove its acquisition basis.
168+
137169
Wallet-only requests skip the unused custody aggregate. On a warm, read-only
138170
production snapshot of 10,540 wallets, this reduced owner-query time from
139171
1.34 s to 0.78 s; the reported single-wallet query fell from 1.65 s to 0.63 s.

0 commit comments

Comments
 (0)