fix(realtime): filtered fallback when a full-block eth_getLogs exceeds viem's body cap (issue #23)

[dzhelezov]

Fixes #23.

The bug (RPC-transport realtime only, inherited upstream)

In RPC-transport realtime, ponder's block-data fetch (fetchBlockEventData in sync-realtime/index.ts) issues an unfiltered eth_getLogs pinned by blockHash — every log in the block. On a "monster" block the response body exceeds viem's hardcoded ~10 MiB readResponseBody cap (viem ≥ 2.31) and viem throws ResponseBodyTooLargeError. Because the request is pinned by blockHash, every retry deterministically hits the same oversized body — the chain stalls, then crash-loops at the tip, surfacing as an unhandledRejection after N futile retries (exit 75). No wrong data is ever stored (a restart backfills past the block with filtered fetches), so this is availability-not-correctness, severity medium.

Both Portal paths are structurally immune — historical backfill fetches filtered eth_getLogs, and PORTAL_REALTIME=stream never issues a full-block log fetch — so this fix is purely for users on RPC-transport realtime.

The fix (issue disposition candidates 1 + 3)

A wiring-patch hunk applied to all four supported versions (0.15.17 / 0.16.6 / 0.16.7 / 0.16.8):

1. Filtered fallback (rpc/actions.ts, three exported/unit-testable helpers)

• isResponseBodyTooLargeError(error) — matches by error.name === "ResponseBodyTooLargeError", walking the .cause chain. Deliberately not instanceof: the class is absent from older pinned viem, and the ponder core floats viem: ">=2", so a name test is robust across both. (Confirmed against the installed viem: 2.30.1 has no such class; a fresh >=2 install resolves 2.55.0 which throws it raw at the call site.)
• buildFilteredBlockLogRequests({logFilters, factories}) — the deduped {address, topic0} set covering every registered log filter and factory. A factory-address filter contributes a wildcard address (its child addresses are only discovered from this block's own logs, so they can't be pre-filtered) and is re-narrowed downstream; each factory contributes {factory.address, factory.eventSelector}. topic1..3 and child-address matching are omitted because the caller re-applies exact matching on the assembled logs — so this only over-fetches (bounded, identical to what historical sync fetches), never under-fetches.
• eth_getFilteredBlockLogs(rpc, blockHash, {logFilters, factories}) — one bounded filtered eth_getLogs per request, assembled and deduped by (blockHash, logIndex), sorted by logIndex.

Interception point (sync-realtime/index.ts) — the unfiltered log fetch is wrapped; on the oversized error it falls back to the filtered assembly for the same block. The block fetch is decoupled from the logs fetch so an oversized logs leg never discards a healthy block. The empty-vs-non-empty-logsBloom check in validateLogsAndBlock is skipped only on the empty-fallback path (the filtered result is legitimately empty on a monster block that matches no registered filter) — mirroring the if (logs.length > 0) guard that historical filtered sync already uses; all per-log consistency checks still run when the fallback returns logs.

2. Non-retryable (rpc/index.ts) — shouldRetry returns false for ResponseBodyTooLargeError, so the deterministic oversized body surfaces immediately instead of after 9 futile retries or as an unhandledRejection.

Known limitation (documented, not a regression)

The filtered fallback is bounded by the registered filters — a strict improvement over today's unconditional crash. A wildcard-address filter on a very hot topic0 (e.g. Transfer) could in principle still exceed the cap in the filtered request; that residual re-throw is now non-retryable and surfaces cleanly rather than crash-looping. As the issue notes, any fixed limit just moves the threshold.

Tests & evidence

Regression test portal/realtime-getlogs-fallback.test.ts — mutation-verified RED on unpatched upstream (the three helpers don't exist and shouldRetry is neither exported nor returns false; with shouldRetry merely exported it returns true for the oversized error, i.e. the bug). Covers: name/cause-chain matching, non-retryable behavior, request-building (static/factory/dedup/union), fetch→assemble→dedup→sort, and the empty-fallback validation hazard the guard sidesteps.

• Both-version scripts/sync-upstream.sh {0.16.8,0.16.6} --test green — 17 test files, 326 tests each (13 new).
• npx @biomejs/biome@2.5.2 check . clean (0 errors).

Reported upstream-worthy: this is inherited upstream ponder realtime behavior.

🤖 Generated with Claude Code