Skip to content

fix: read events when the node's sdk drifts from the runtime's - #1503

Merged
mrq1911 merged 7 commits into
polkadot-stable2603from
fix/synth-event-decode-2603
Jul 31, 2026
Merged

fix: read events when the node's sdk drifts from the runtime's#1503
mrq1911 merged 7 commits into
polkadot-stable2603from
fix/synth-event-decode-2603

Conversation

@mrq1911

@mrq1911 mrq1911 commented Jul 30, 2026

Copy link
Copy Markdown
Member

Three bugs in the node-side synthetic-logs layer, plus an envelope rework so synth txs are legible to ordinary EVM tooling.

1. Node/runtime sdk skew silently zeroed synth logs (live on mainnet)

v49.2.0 is a node-only release on polkadot-stable2603 while the runtime stays on stable2506. Upstream polkadot-sdk#7250 inserted MintedCredit(11) and BurnedDebt(13) into the middle of pallet_balances::Event, shifting Burned 11→12, Issued 15→17, Rescinded 16→18, Locked 17→19, Frozen 19→21.

Synth logs decode System::Events with the node's compiled RuntimeEvent, so the chain's Issued{amount} (16 B) is read as Restored{who,amount} (48 B) → the whole Vec<EventRecord> decode fails → Decode::decode(..).ok() turns that into Vec::new() and the block looks event-free. No log, no metric, no RPC error.

Measured on mainnet: rpc.hydradx.cloud and hydration-rpc.n.dwellir.com return 0 logs for every block paying fees in native HDX (~18% of blocks), including the wormhole core-bridge LogMessagePublished raised by substrate-dispatched EVM.call — so guardians never observed NTT DAI seq 2–5.

Fixcompat_events::read_events, in descending fidelity:

  • Native → typed decode, unchanged fast path, free when node and runtime agree
  • Compat → explicit per-sdk pallet_balances::Event layout, other pallets via the node's own types → exact
  • Partial → resync + a whole-blob EVM.Log sweep (see §3)

2. Tx-hash domain keyed on the parent hash → collisions on every fork

block_domain folded parent_hash + block_number. Sibling blocks at the same height share a parent, so every field feeding keccak(RLP(tx)) was identical for both — two blocks minting the same synth tx hash, which frontier indexes by hash. On a chain whose fork rate warranted an emergency release, that is live. Now keyed on the block's own hash, distinct by construction.

3. Partial lost runs of records, and did so silently

Two compounding defects, both found by replaying block 13384676 (the DAI transfer that published wormhole sequence 2) on a skewed node:

  • Mis-consumption, not skipping. A wrong-layout record still parses while consuming the wrong byte count, swallowing whatever follows. That block's 50 events collapsed into 27 records with only 6 reported skips, and 12 of 19 logs vanished — including the LogMessagePublished. The record itself was intact at byte offset 1710 (phase=ApplyExtrinsic(3), pallet=0x5a, variant=0x00); a mis-decoded neighbour ate it.
  • Shared latch. Compat and Partial shared one Once, so once a recoverable skew reported itself, all subsequent real data loss went unreported.

There is also a second sdk skew beyond balances: that block carries ConvictionVoting.Voted and Scheduler.{Scheduled,Canceled}, whose layouts also moved between stable2506 and stable2603. Per-pallet compat layouts are therefore whack-a-mole and cannot be a guarantee.

Fix — a whole-blob sweep for EVM.Log records, deduped and merged into whatever the sequential walk produced. It works because an EVM.Log record's bytes depend on nothing that skews: phase ‖ pallet ‖ variant ‖ address ‖ topics ‖ data ‖ record_topics. Guards: pallet byte must be this runtime's EVM index, variant must be Log, ≤4 log topics, empty record-topic vec. Partial now reports on its own counter with power-of-two backoff and carries a recovered count.

This guarantees the wormhole class specifically. Other records (e.g. ERC-20 Transfer) can still be incomplete on a doubly-skewed node — only metadata-driven decoding, or a node whose runtime matches the chain, closes that gap.

4. Envelope rework

field before now
from 0xdeadbeef…beef 0x73796e74680000000000000000000073796e7468 — ascii synth..........synth
to sentinel origin account (from TransactionFeePaid), sentinel when there is none
value group index 0 — nothing should see a phantom native transfer
nonce bucket_nonce, up to 2^64-4 (i64::MAX) (block << 16) | group — monotonic, unique per (block, group), int64-safe
input marker ‖ parent_hash ‖ block_number abi calldata hydrationSynthV1(bytes32,bytes32,uint64), selector 0xf1f8393a

input is now an ordinary EVM call, so ethers/viem/cast decode it with the signature alone and explorers with a 4byte lookup render it as a named call; versioning rides on the selector, so a layout change makes stale decoders fail loudly instead of misreading bytes. Verified live with cast decode-calldata, both bytes32 values cross-checked against the substrate side.

Rationale for the rest: the old hook nonces overflow the bigint column indexers keep nonces in (and ~92% of synth txs are hook buckets); a repeating (from, nonce) breaks it as a key and trips replacement-detection; from stays the marker so no unsigned tx is attributed to a user and no real account's nonce sequence is entangled; extrinsicHash in input makes substrate↔EVM correlation a field read. Hashing OpaqueExtrinsic needs no RuntimeCall decode, so none of it is exposed to §1.

Version bump

hydradx-runtime 429→430 + spec_version 429→430, because the fix touches runtime/hydradx/src/evm/ and version-check requires modified crates bumped with runtime major == spec_version. Bookkeeping only — the synth path is native code in the node binary (the runtime API was removed in 9c4da722c), so this still ships as a node-only release. Demonstrated: the fixed node reports compiled runtime 429 while executing chain spec 433 and serves the fixed logs.

Verification

Tests — 19 synthetic_logs, 11 event_logs, 6 node = 36. Notably node_balances_layout_is_known (fails on the next mid-enum event insertion), synth_selector_matches_signature, and wormhole_event_survives_undecodable_neighbours — which replays real mainnet events from block 13384676 and asserts sequence 2 comes out.

On a real node — built this branch, ran it against the pruned mainnet snapshot, synced to head:

  • all four original NTT blocks captured: seq 2, 3, 4, 5 with their full event sets
  • A/B vs rpc.hydradx.cloud: blocks recovered with 2–113 logs each, never fewer
  • vs a stable2506 node as ground truth: 900/900 blocks identical, 0 missed, 0 extra
  • 20/20 synth txs resolve via eth_getTransactionByHash + receipt and appear in block.transactions

Not a substitute for putting eth-rpc nodes on a release whose runtime matches the chain (v50.0.0 today).

@github-actions

Copy link
Copy Markdown

Weight Diff Report

⚠️ 2 pallet(s) have changes exceeding ±10% threshold

35 extrinsic(s) changed across 2 pallet(s). New: 4. Removed: 2.

pallet_omnipool_liquidity_mining

Extrinsic RefTime Proof Size Reads Writes
create_global_farm -0.1% (119.1M → 118.9M) 8 → 7 (-1)
update_global_farm +0.8% (164.2M → 165.6M) 9 → 8 (-1)
terminate_global_farm +1.5% (137.6M → 139.6M) 8 → 7 (-1)
create_yield_farm -0.6% (186.5M → 185.5M) 12 → 11 (-1)
update_yield_farm -0.1% (191.7M → 191.6M) 12 → 11 (-1)
stop_yield_farm -0.1% (184.5M → 184.4M) 11 → 10 (-1)
resume_yield_farm +0.2% (188.5M → 188.9M) 12 → 11 (-1)
terminate_yield_farm -0.9% (135.8M → 134.6M) 9 → 8 (-1)
deposit_shares -0.3% (282.1M → 281.3M) 20 → 19 (-1)
redeposit_shares -0.1% (242.5M → 242.3M) 18 → 17 (-1)
claim_rewards +1.5% (244.9M → 248.5M) 14 → 13 (-1)
withdraw_shares +0.2% (336.4M → 337.0M) 19 → 18 (-1)
join_farms -2.3% (172.4M → 168.4M) 16 → 15 (-1)
add_liquidity_and_join_farms -2.2% (517.9M → 506.5M) 46 → 45 (-1) 25 → 26 (+1)
exit_farms +0.8% (107.7M → 108.6M) 13 → 12 (-1)
add_liquidity_stableswap_omnipool_and_join_farms ⚠️ -22.5% (2.7B → 2.1B) 80 → 77 (-3)
remove_liquidity_stableswap_omnipool_and_exit_farms ⚠️ -35.2% (1.7B → 1.1B) 64 → 63 (-1)
price_adjustment_get -1.8% (177.3M → 174.2M)

pallet_stableswap

Extrinsic RefTime Proof Size Reads Writes
create_pool +0.6% (54.4M → 54.7M)
create_pool_with_pegs -0.8% (100.2M → 99.4M)
add_assets_liquidity -0.5% (2.8B → 2.8B) 54 → 53 (-1) 18 → 17 (-1)
add_liquidity_shares -1.1% (2.1B → 2.1B) 38 → 37 (-1) 10 → 9 (-1)
remove_liquidity_one_asset -1.3% (2.1B → 2.1B) 35 → 34 (-1) 9 → 8 (-1)
remove_liquidity -0.4% (1.7B → 1.7B) 56 → 55 (-1) 14 → 13 (-1)
withdraw_asset_amount -4.3% (2.6B → 2.5B) 36 → 35 (-1) 8 → 7 (-1)
sell -1.0% (2.1B → 2.1B)
buy -1.1% (2.0B → 2.0B)
set_asset_tradable_state +6.5% (23.1M → 24.6M) 1 → 2 (+1)
update_pool_fee -2.0% (21.4M → 21.0M)
update_amplification -2.3% (22.4M → 21.9M)
update_asset_peg_source -2.9% (28.7M → 27.9M)
update_pool_max_peg_update -1.8% (26.1M → 25.6M)
router_execution_sell +0.2% (1.2B → 1.2B) ⚠️ +18.6% (13990 → 16590) 18 → 19 (+1)
router_execution_buy -0.5% (588.8M → 585.6M)
calculate_spot_price_with_fee +3.5% (494.3M → 511.8M) ⚠️ +18.6% (13990 → 16590) 17 → 18 (+1)
New extrinsics (4)
Pallet Extrinsic RefTime Proof Reads Writes
cumulus_pallet_parachain_system block_weight_tx_extension_max_weight 0 0 0 0
cumulus_pallet_parachain_system block_weight_tx_extension_stays_fraction_of_core 0 0 0 0
cumulus_pallet_parachain_system block_weight_tx_extension_full_core 0 0 0 0
pallet_xcm weigh_message 100.0K 0 0 0
Removed extrinsics (2)
Pallet Extrinsic RefTime Proof Reads Writes
pallet_evm_accounts set_ntt_minter 23.3M 0 0 1
pallet_evm_accounts clear_ntt_minter 23.2M 0 0 1

Threshold: ±10%. Base Weight::from_parts(ref_time, proof_size) compared; per-unit components ignored.

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Crate versions that have been updated:

  • hydradx: v16.0.0 -> v16.0.1
  • hydradx-runtime: v429.0.0 -> v430.0.0

Runtime version has been increased.

@mrq1911
mrq1911 merged commit d7fd854 into polkadot-stable2603 Jul 31, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant