eth/catalyst, eth/ethconfig, cmd: make engine API max reorg depth configurable

[danceratopz]

TLDR

The enginex simulator runs multiple tests against a single client instance, for each test, the simulator FCUs to genesis and then after each payload in the test: This triggers reorgs larger than currently supported by geth. More details in the enginex docs. The simulator's FCU behavior was designed to be consistent with the original engine simulator; an alternative would be to change the simulator to only verify payloads without FCU, but I think it's cleaner as it is currently (per docs above).

Feel free to close this PR/re-implement as you see fit; whatever's easiest. Thanks!

Motivation

#34767 (implementing execution-apis#786) introduced a hardcoded cap of 32 on the depth of chain rewinds accepted via engine_forkchoiceUpdated when the target block is an already-canonical ancestor. This breaks systematic testing with EEST's consume enginex simulator (via hive):

• enginex reuses one client across many tests within a pre-allocation group, and each test starts by sending a forkchoiceUpdated with head = genesis to reset the shared client.
• That reset is a canonical-ancestor rewind whose depth equals the chain height left by the previous test. As soon as one test's chain reaches height >= 32, the reset is refused with -38006 Too deep reorg (e.g. reorg depth 37 exceeds limit 32).
• Because the refused rewind leaves the head in place, every subsequent test on that client fails identically — a single deep test permanently poisons the shared client.

Erigon has the same cap but reads it from the MAX_REORG_DEPTH environment variable, which is how hive already works around it for enginex runs (ethereum/hive#1568, raising it to 512). Geth currently offers no knob.

Changes

• New flag --engine.maxreorgdepth (default: 32, 0 = no limit), wired through ethconfig.Config.EngineMaxReorgDepth following the --rpc.gascap pattern. Since all geth flags are auto-exposed as environment variables, it can also be set via GETH_ENGINE_MAXREORGDEPTH; older geth versions ignore unknown GETH_* variables with a warning, so test runners can set it unconditionally.
• catalyst.ConsensusAPI reads the limit from the config at construction instead of the package-level constant. A limit of 0 disables the check, matching the --rpc.gascap "0 disables" convention (and pre-eth/catalyst: allow reorging the head block to a parent #34767 behavior). Default CLI behavior is unchanged (32).
• Added TestForkchoiceUpdatedReorgDepthLimit covering refusal beyond the configured limit, acceptance within it, and unlimited rewind-to-genesis with the check disabled. startEthService gained optional config-modifier arguments; existing callers are untouched.

Notes

• core/blockchain.go's reorg path still logs Impossible reorg, please file an issue (error level) for canonical rewinds deeper than its hardcoded 32 heuristic, even when the operator has raised the engine API limit; the rewind itself proceeds correctly. Left as-is to keep this change small — happy to plumb the configured limit into BlockChainConfig in this PR or a follow-up if preferred.
• Hive-side companion (required for enginex runs to benefit): clients/go-ethereum: raise engine API max reorg depth for enginex runs hive#1572 sets GETH_ENGINE_MAXREORGDEPTH=512 in clients/go-ethereum/geth.sh when the simulator requests deep reorgs, mirroring the erigon fix in erigon: raise MAX_REORG_DEPTH for enginex runs hive#1568.

Verification

Verified against EEST's consume enginex in hive --dev mode, using the minimal failing pair: a 37-block test followed by any test on the same shared client (the second test's reset-to-genesis forkchoiceUpdated is a depth-37 canonical rewind).

Setup, in a hive checkout: rsync this branch to clients/go-ethereum/go-ethereum-local/ (exclude .git), add export GETH_ENGINE_MAXREORGDEPTH=512 to clients/go-ethereum/geth.sh (this is what ethereum/hive#1572 does, gated on the simulator opting in — that PR is required for enginex runs to pick this fix up), and create geth-local.yaml:

- client: go-ethereum
  nametag: maxreorg-local
  dockerfile: local
  build_args:
    local_path: go-ethereum-local

Fastest: run hive in --dev mode and drive it with consume from an execution-specs (EEST) checkout, filling the two fixtures locally instead of downloading the release:

./hive --dev --client go-ethereum --client-file geth-local.yaml \
  --docker.output --dev.addr 127.0.0.1:3011

uv run fill --generate-all-formats -m blockchain_test_engine_x --output=fixtures/ --clean \
  --regex='.*fork_Osaka-blockchain_test_engine_x-test_program_program_(BALANCE|CALLER)-debug.*' \
  tests/frontier/scenarios/ -v

export HIVE_SIMULATOR=http://127.0.0.1:3011
uv run consume enginex --input=fixtures/ -v

Alternatively, run the two tests with a single standalone hive command (the first run builds the simulator image, which downloads the tests@v20.0.0 fixture release):

./hive --sim ethereum/eels/consume-enginex \
  --sim.buildarg fixtures=tests@v20.0.0 \
  --client go-ethereum --client-file geth-local.yaml \
  --sim.limit '.*fork_Osaka-blockchain_test_engine_x-test_program_program_(BALANCE|CALLER)-debug.*' \
  --docker.output

Results (identical in both modes):

• Control (this branch, GETH_ENGINE_MAXREORGDEPTH unset, default 32 — identical to current master): BALANCE passes leaving the head at block 37, CALLER fails its initial forkchoice update with -38006 Too deep reorg: reorg depth 37 exceeds limit 32.
• With GETH_ENGINE_MAXREORGDEPTH=512: client logs Set engine API maximum reorg depth depth=512; both tests pass.

[jrhea]

sgtm

[rjl493456442]

Sgtm, I guess engine.* is an appropriate flag prefix for engine API, and I am pretty sure we will pile up more in the future, with SSZ engine API landing.