refactor: decompose combined retrieval into orthogonal components#64
Merged
Merged
Conversation
`CombinedCalculator` is now a thin orchestration façade over four focused submodules: `TransferLogScanner` decodes Transfer events out of `eth_getLogs`, `TxReceiptEnricher` fetches transactions and receipts (carrying the zkSync permissive raw-decode fallback and the bounded serial-retry loop), `extract_gas_and_amount` is a pure function that turns each (tx, receipt) pair into a `GasAndAmountForTx`, and the shared `failure` module owns lookup-attempt/lookup-failure construction and the operator-facing skip log. The public surface — `new`, `with_config`, `calculate_combined_data_with_adapter`, and the Ethereum/Optimism convenience methods — and the partial-failure metadata callers depend on are unchanged on the wire. The motivation is that combined retrieval was previously a ~1600-line file mixing scan, lookup, gas extraction, retry policy, aggregation, and tracing. Each step is now independently unit-testable: the scanner decode loop runs against synthetic logs without a provider, the gas extractor exercises every (tx-present, receipt-present, transport error) arm with synthetic inputs, and the failure helpers cover the error-chain walk and transport-source extraction directly. The existing end-to-end integration tests around batch + serial + zkSync behaviour stay in `calculator.rs` and continue to gate the wired pipeline. Closes #24.
Contributor
Author
Self-review passPre-commit
No blockers, no follow-up issues filed. The PR shipped with a single atomic commit; the two |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CombinedCalculatoris now a thin orchestration façade over four focused submodules. The public surface (constructors andcalculate_combined_data_*methods) and the partial-failure metadata callers depend on are unchanged on the wire; this is an internal layout change that exposes smaller seams for future work. Closes #24.What's in the diff
src/retrieval/transfer_log_scanner.rs(new) — owns theTransferfilter build, theLogScannerinvocation, and the raw-RpcLog→LogBatchEntrydecode loop. The decode loop is its ownpub(crate) fnso it is unit-testable without a provider.src/retrieval/tx_receipt_enricher.rs(new) — ownseth_getTransactionByHash+eth_getTransactionReceiptfetching (parallel viatokio::join!), the bounded serial-retry loop, and the zkSync permissive raw-decode fallback. Carries theprocess_log_for_combined_dataspan at the same boundary the original did. Migrates theshould_attempt_permissive_tx_decodepredicate test.src/retrieval/gas_extractor.rs(new) — pureextract_gas_and_amountfree function plus theTransactionGasDatahelper. No I/O; new unit tests cover the missing-tx, missing-receipt, transport-error, and gas-price-override arms.src/retrieval/failure.rs(new) — sharedcollect_error_chain,transport_error_string,build_lookup_attempt,build_lookup_failure,lookup_request_failed,log_combined_data_skip. Unit tests cover the pure helpers.src/retrieval/calculator.rs— slimmed from ~1600 to ~750 lines (production code now ~280); reads as orchestration. All existing integration tests (zkSync raw fallback, tx/receipt failure→partial, fallback recovery, zero-fallback config, zkSync missing-access-list happy path) stay here and continue to gate the wired pipeline.src/retrieval/mod.rs— declares the four new submodules; no new public re-exports.Reviewers might want to start at the new
calculator.rsand trace into the components.Acceptance check (from #24)
calculator.rsis substantially smaller and primarily wires components together — production code dropped from ~795 to ~280 lines.decode_transfer_logsintransfer_log_scanner.rswith four synthetic-log tests.enrich_batchandretry_faileddirectly; the predicate test moved with the predicate.gas_extractor.rshas five unit tests against synthetic inputs.calculator.rsplus the new component-level unit tests are green across--all-features, default, and--no-default-features.examples/zksync_combined_probe.rsstill compiles —cargo check --examples --all-featuresclean.failuremodule unchanged; thepass/stage/attemptsaccumulation behaviour matches.Test plan
cargo test --all-features— 494 lib tests + integration tests + doctests passcargo test(default features) — 491 lib tests + integration tests + doctests passcargo test --no-default-features— 491 lib tests + integration tests + doctests passcargo clippy --all-targets --all-features -- -D warningscleancargo fmt --checkcleancargo check --examples --all-featurescleanreuse lint— 104/104 files compliant