Support optional execution and proof engines in optimistic sync

[frisitano]

Summary

Alternative direction to #5151 for the optional-engine aspect of EIP-8025. A single-file prose addition to sync/optimistic.md that handles engine-absence and ProofEngine-as-fork-choice-signal entirely via existing optimistic-sync semantics, with no function-signature changes, no modification to beacon-chain.md pseudocode, and no new validator.md.

Two rules added to sync/optimistic.md

1. Engine absence → NOT_VALIDATED. If a consensus engine is not configured with an ExecutionEngine (or ProofEngine), the corresponding signal is treated as NOT_VALIDATED for optimistic-import purposes. A block is non-optimistic only after at least one signal source returns VALID.

2. ProofEngine as parallel VALID signal source. When the ProofEngine returns VALID for a SignedExecutionProof whose public_input.new_payload_request_root matches a block's NewPayloadRequest, the consensus engine transitions the block from NOT_VALIDATED → VALID, parallel to the existing ExecutionEngine VALID transition. The ProofEngine is not a source of INVALIDATED transitions — a non-VALID proof response is handled at gossip-validation and does not constitute evidence that the execution payload itself is invalid.

Why this direction

• Keeps ExecutionEngine and ProofEngine strictly disjoint at the function level. No coupling inside verify_and_notify_new_payload, contra @mkalinin's earlier sketch in Make ExecutionEngine highly recommended but optional frisitano/consensus-specs#3. Two orthogonal validity sources stay orthogonal.
• No Optional[] cascade through the fork hierarchy. process_execution_payload's signature is unchanged. Every call site in tests and helpers is unaffected.
• sync/optimistic.md is the idiomatic home. The file is already the spec authority on NOT_VALIDATED semantics and signal-driven state transitions. Adding a second signal source (PE) and an engine-absence rule fits the existing pattern.
• Existing optimistic-sync attestation rules already cover honest-validator behaviour. A block in NOT_VALIDATED state by the attestation deadline is not attestable — this rule is already in place, so no bespoke validator.md is needed.

Relationship to #5151

File-level compatible (no overlap). Architecturally alternative:

• Make ExecutionEngine highly recommended but optional #5151 removes the EE assert in process_execution_payload and adds a new validator.md with a SHOULD-rule on attestation.
• This PR keeps the EE assert, keeps the beacon-chain.md pseudocode as-is, and adds no new validator file. The semantic work happens in optimistic.md prose.

If this direction is preferred, #5151 could be reduced in scope to just *Note* wording fixes (if any) or closed in favour of this PR.

Prior discussion

frisitano#3 contained the original review thread on the pre-#5055 version. Key comments there:

• @frisitano on line 120: wording on "accepts execution payloads" should acknowledge NOT_VALIDATED / optimistic import semantics.
• @frisitano on line 76 of validator.md: observed overlap between the proposed validator rule and existing Optimistic Sync attestation rules.
• @mkalinin on line 155: proposed keeping the assert and coupling EE + PE inside verify_and_notify_new_payload, routing through Optimistic Sync. This PR takes the Optimistic Sync direction but without the EE/PE coupling.

Test plan

• Rendered markdown reads correctly on GitHub (ToC, anchor to process_execution_proof in beacon-chain.md).
• Reviewer alignment with @barnabasbusa, @mkalinin on which architectural direction to take (this PR's optimistic-sync-only approach vs Make ExecutionEngine highly recommended but optional #5151's beacon-chain.md + validator.md approach).
• Confirm no interaction with existing optimistic-sync conformance tests.

🤖 Generated with Claude Code

[mkalinin]

How then any node that relies solely on ProofEngine can invalidate an invalid block? If this is impossible then one of potential problems for a node would be to be locked in the optimistic state forever without ability to produce blocks and attest unless a valid block that reorgs out an invalid one is proposed by someone else in the network.

[frisitano]

Yes, this is correct. For optional proofs, we strongly recommend that clients have an EL to address such issues. We expect only a small subset of validators to operate without an EL, and they should be aware of the risks this creates.

For mandatory proofs, this is not a problem; if valid proofs are not made available in a timely fashion, the payload is considered absent for that slot.

I would suggest we do not overfit on a configuration (no EL) that we do not recommend and instead leave this issue to be resolved in mandatory proofs (it seems to be an edge case anyway). Keen to hear your thoughts.

[mkalinin]

I agree, just wanted to check on the ways of how this is going to be resolved. This is a payload status change, with proofs we are loosing INVALID status with all possible outcomes. So, SYNCING on an invalid chain won’t ever resolve and we have to introduce a timeout — the thing that we don’t have before — to resolve from that situation. Otherwise, a network will stay in limbo in the case of invalid payload.

Today SYNCING signals that EL is attempting to pull the data from p2p and execute ancestor payloads to obtain the state required for validation of a given payload. And SYNCING -> INVALID is possible as the validation of a payload or one of its ancestors may fail. It may be that some ancestor payload doesn’t exist in the network anymore (it was invalid and all peers dropped that one) and then SYNCING will never resolve without switching to an available chain. And that switch may happen because some validators are online at this stage and are able to produce a block reorging out an INVALID payload.

The absence of the proof is similar to data required for payload validation being unavailable. I believe that the presence of an invalid proof is quite similar to the absence of the proof. And here we need online nodes to reorg a payload that doesn’t have a proof by the end of the slot, I believe.

[frisitano]

I agree, just wanted to check on the ways of how this is going to be resolved. This is a payload status change, with proofs we are loosing INVALID status with all possible outcomes. So, SYNCING on an invalid chain won’t ever resolve and we have to introduce a timeout — the thing that we don’t have before — to resolve from that situation. Otherwise, a network will stay in limbo in the case of invalid payload.

Technically speaking, I believe we do, in fact, have proof of invalid payloads. If I'm not mistaken, this is something that @jsign added to the proof API. If we want to integrate this at the protocol level, we can.

Today SYNCING signals that EL is attempting to pull the data from p2p and execute ancestor payloads to obtain the state required for validation of a given payload. And SYNCING -> INVALID is possible as the validation of a payload or one of its ancestors may fail. It may be that some ancestor payload doesn’t exist in the network anymore (it was invalid and all peers dropped that one) and then SYNCING will never resolve without switching to an available chain. And that switch may happen because some validators are online at this stage and are able to produce a block reorging out an INVALID payload.

The absence of the proof is similar to data required for payload validation being unavailable. I believe that the presence of an invalid proof is quite similar to the absence of the proof. And here we need online nodes to reorg a payload that doesn’t have a proof by the end of the slot, I believe.

Yes, that's what I had in mind. Even if we introduce proofs for invalid payloads, I believe the outcome will be the same as if the proofs were unavailable regarding validator attestation.