feat(evm): lower the status-check retry-delay floor from 5s to 1s

[dylankilkenny]

Problem

#870 (implemented in #847) lets each network set the status-check retry interval. The minimum is 5 seconds. We want a minimum of 1 second for fast chains.

The minimum is 5 for one reason. A status check does more than poll the chain. It can also run nonce reconciliation, evaluate resubmission, and update the circuit-breaker counters. The old fixed 8–12s backoff limited how often these three could run. A 1-second interval removes that limit. Each of the three needs its own limit before the minimum can drop.

Work items

1. Bound nonce reconciliation (S)

Today, a hinted status check runs reconciliation on every retry delivery (src/domain/transaction/evm/status.rs, TX_NONCE_RECONCILE_TRIGGER). At 1s, that is several RPC calls per second. Bound it by attempt:

const MAX_NONCE_RECONCILE_ATTEMPTS: u32 = 3;
// gate: ctx.total_retries < MAX_NONCE_RECONCILE_ATTEMPTS

Do not bound it to attempt 0. A transient failure on the first delivery must not stop reconciliation.

Prerequisite: the three handlers that share transaction_status_handler must count attempts the same way (src/queues/redis/worker.rs). Normalize in one place.

2. Give resubmission its own clock (S)

At 1s, a stuck transaction is re-priced, re-signed, and re-broadcast every resend window. #847 tried to fix this: it refreshed sent_at on duplicate-submit. But hash recovery also reads sent_at. On Arbitrum, the fixed 20s resubmit window reset the clock before the 2-minute recovery gate could open, so recovery stopped.

Fix: add a last_broadcast_at field. Resubmission reads and resets it. Recovery keeps reading sent_at. The resubmit path never touches sent_at.

3. Make the circuit breaker count time, not checks (M)

The breaker gives up after N failed checks in a row. At 1s, N checks pass ~10x faster than at 8–12s, so the breaker gives up ~10x sooner. Fix: give up after X minutes of failure, at any check speed. This also helps #871.

Sub-item, can land first as its own small PR: do not force-fail a Mined transaction (the is_circuit_breaker_remediable change from #847). A transaction that is on chain must not be marked failed. This bug exists at any check speed.

4. Drop the minimum (trivial, last)

Change MIN_EVM_STATUS_CHECK_RETRY_DELAY_SECONDS from 5 to 1. Load-test on a fast chain first: N pending transactions at 1s = N RPC checks per second.

Non-goals

• The initial-delay minimum stays 1.
• refactor(queues): self-scheduling status checks — remove the pending-tx concurrency ceiling #871 (self-scheduling checks) is separate. Work item 3 helps it.