refactor(forks): move verify_signatures body into the spec (Stage 4C, part 1 of leanEthereum#686) (leanEthereum#704)

* refactor(forks): move verify_signatures body into the spec class

Moves the full XMSS signature verification logic from
SignedBlock.verify_signatures into LstarSpec.verify_signatures.
SignedBlock becomes a pure SSZ data container.

Internal callers that still hold a Store (notably Store.on_block,
which itself moves to the spec class in a follow-up) reach the
verification path via a deferred import of LstarSpec to sidestep
the spec ↔ store module-load cycle.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): move State and Store bodies into the spec class

Migrates every State and Store method body into LstarSpec. Containers
(State, Store, SignedBlock) become thin Pydantic data classes whose
methods are one-line forwarders to the active fork spec, reached via a
deferred import that breaks the spec ↔ container module-load cycle.

Inside the moved bodies, every literal Block(...), BlockBody(...),
BlockHeader(...), Config(...), AggregatedAttestations(...), and other
container constructor is now self.<name>_class(...). An inheriting fork
that swaps a single container type therefore receives the parent fork's
logic for free.

Observability hooks (observe_state_transition, observe_on_block,
observe_on_attestation) ride along with the bodies, preserving the
metrics surface.

The obsolete delegator-forwarding test file is removed; behavioural
coverage now lives in the existing state-transition, fork-choice, and
block-production test suites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): delete the trivial container forwarders (Stage 4D of leanEthereum#686)

State, Store, and SignedBlock become pure Pydantic data containers.
All forwarder methods that delegated through the lazy spec singleton
are removed; the lazy singleton helpers are removed alongside them.

Every remaining call site that used to go through a container method
now goes through the active fork spec:

- Tests use the session-scoped `spec` fixture from `tests/lean_spec/conftest.py`.
- Subspec services (`sync`, `validator`, `chain`, plus the fork-choice
  API endpoint) carry a module-level `_SPEC = LstarSpec()` constant.
- Library helpers (`tests/lean_spec/helpers/builders.py`,
  `packages/testing/...`) follow the same `_SPEC` pattern.

`ForkProtocol.generate_genesis` and `ForkProtocol.create_store` become
abstract; the previous default implementations referenced container
methods that no longer exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): resolve Block forward reference in BlockLookup

The Store.blocks field is annotated as `BlockLookup`, which was defined
as `dict[Bytes32, "Block"]` — a string forward reference. After the
container methods moved off Store, Pydantic's model rebuild could no
longer resolve `Block` because it was never imported into store.py
alongside the alias.

Drop the forward-reference quoting: `BlockLookup` lives in the same
module as `Block`, so the type can refer to the class directly. Pydantic
then resolves `Store.blocks` correctly through the alias re-export.

Without this, every consensus filler that constructed a Store via
`spec.create_store(...)` raised
`PydanticUserError: 'Store' is not fully defined`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): route mock-store tests through autouse spec patches

The forwarders that used to live on `Store` (and were patched in
`tests/lean_spec/subspecs/{sync,chain,networking,validator}/`) are gone
after Stage 4D. The mocks (`MockStore`, `MockForkchoiceStore`) still
implement the same method surface, but the service code now calls the
real spec, which expects a Pydantic Store.

Add an autouse fixture per affected subspec that patches the active
spec's methods to delegate back to `store.method(...)`. The mocks
intercept calls in-place, preserving every test's recording semantics
without touching service code.

The validator service tests that previously patched
`Store.produce_block_with_signatures` and `Store.on_gossip_attestation`
now patch the matching methods on the validator service's `_SPEC` —
same intent, current attribute path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): drop cast(Store, ...) at call sites

Casts were a workaround for Liskov violations on LstarSpec.create_store
when it returned the SpecStoreType protocol while concretely producing
Store. cast had no runtime effect and pushed type-checker noise into
fixtures and tests.

Move the imprecision into the fork itself: create_store now declares
its concrete Store return and suppresses the override warning at the
single definition site. Callers receive Store directly and need no cast.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): replace 17-site _SPEC duplication with DI

Each module declared its own _SPEC = LstarSpec(), creating value-equal
but identity-distinct instances. The pattern also baked LstarSpec into
17 import sites; a future fork would have to grep-and-replace them all.

Production services (chain, sync, validator, api) now take spec as a
dataclass field with default_factory=LstarSpec — explicit at the
composition root in node.py, optional in tests. node.py narrows
config.fork (ForkProtocol) to LstarSpec once with isinstance, which
also lets the cast(State, ...) and cast(Store, ...) at the genesis
construction sites drop.

Test conftests that intercept spec calls now monkey-patch LstarSpec
at the class level (not the deleted module-level _SPEC instance).
Test types and fixtures instantiate LstarSpec at call time — no
module-level cache, no shared mutable state to alias.

ForkProtocol still declares only the three abstract construction
methods (generate_genesis, create_store, upgrade_state). Services and
tests that drive consensus methods (process_slots, build_block,
tick_interval, ...) keep the concrete LstarSpec type until the
protocol surface is widened in a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(tests): replace conftest monkey-patches with injected spec

Three autouse fixtures in chain/sync/networking conftests patched
LstarSpec class methods so MockStore / MockForkchoiceStore could
intercept consensus calls in place. Class-level patching mutates
shared state and runs against every test in the directory whether
needed or not.

Now that services accept a spec field, tests inject a small
StoreInterceptingSpec subclass that forwards each spec call back to
the store argument. make_store() (used by sync/networking tests)
hands the intercepting spec to the real SyncService transparently.
Chain test_service.py threads it through ChainService directly.

Two conftests delete entirely; the sync conftest keeps only its
sample_checkpoint / sample_status fixtures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): patch validator spec on the instance, not the dropped module

Two validator tests still resolved `lean_spec.subspecs.validator.service._SPEC`,
which was removed when the spec moved onto the service as a field. Patching now
targets `service.spec` directly via `patch.object`, which also exercises the
single instance the test actually calls into.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: tcoratger

packages/testing/src/consensus_testing/genesis.py

@@ -4,18 +4,12 @@
 from lean_spec.forks.lstar.containers.state import State, Validators
 from lean_spec.forks.lstar.containers.validator import Validator
 from lean_spec.forks.lstar.spec import LstarSpec
-from lean_spec.forks.protocol import ForkProtocol
 from lean_spec.subspecs.ssz.hash import hash_tree_root
 from lean_spec.types import Bytes52, Slot, Uint64, ValidatorIndex
 
 from .keys import XmssKeyManager
 
 _DEFAULT_GENESIS_TIME = Uint64(0)
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all helper invocations."""
-
-_DEFAULT_FORK: ForkProtocol = _SPEC
-"""Stateless fork instance used when callers do not pass one explicitly."""
 
 
 def _build_validators(num_validators: int) -> Validators:
@@ -44,30 +38,30 @@ def _build_validators(num_validators: int) -> Validators:
 
 
 def generate_pre_state(
-    fork: ForkProtocol = _DEFAULT_FORK,
+    fork: LstarSpec | None = None,
     genesis_time: Uint64 = _DEFAULT_GENESIS_TIME,
     num_validators: int = 4,
 ) -> State:
     """Generate a default pre-state for consensus tests.
 
     Args:
-        fork: Fork dispatching genesis construction.
+        fork: Fork dispatching genesis construction. Defaults to a fresh
+            LstarSpec instance.
         genesis_time: The genesis timestamp.
         num_validators: Number of validators to include.
 
     Returns:
         A properly initialized consensus state.
     """
+    fork = fork or LstarSpec()
     validators = _build_validators(num_validators)
-    state = fork.generate_genesis(genesis_time=genesis_time, validators=validators)
-    assert isinstance(state, State)
-    return state
+    return fork.generate_genesis(genesis_time=genesis_time, validators=validators)
 
 
 def build_anchor(
     num_validators: int,
     anchor_slot: Slot,
-    fork: ForkProtocol = _DEFAULT_FORK,
+    fork: LstarSpec | None = None,
     genesis_time: Uint64 = _DEFAULT_GENESIS_TIME,
 ) -> tuple[State, Block]:
     """Build a consistent non-genesis anchor by advancing the genesis state.
@@ -101,6 +95,7 @@ def build_anchor(
             "For a genesis anchor use generate_pre_state instead."
         )
 
+    fork = fork or LstarSpec()
     state = generate_pre_state(fork=fork, genesis_time=genesis_time, num_validators=num_validators)
 
     # Reconstruct the genesis block from the state's latest header.
@@ -124,7 +119,7 @@ def build_anchor(
     for next_slot in range(1, int(anchor_slot) + 1):
         slot = Slot(next_slot)
         proposer_index = ValidatorIndex(int(slot) % int(num_validators_u64))
-        current_block, state, _, _ = _SPEC.build_block(
+        current_block, state, _, _ = fork.build_block(
             state,
             slot=slot,
             proposer_index=proposer_index,

packages/testing/src/consensus_testing/test_fixtures/api_endpoint.py

@@ -53,7 +53,7 @@ def _build_store(num_validators: int, genesis_time: int, anchor_slot: int = 0) -
         )
         block = _make_genesis_block(state)
         # No validator identity — fixture only reads store data, never signs.
-        return Store.from_anchor(state, block, validator_id=None)
+        return fork.create_store(state, block, validator_id=None)
 
     # Walk the chain from genesis through anchor_slot using empty blocks.
     # The returned pair (state, block) is internally consistent with the
@@ -64,7 +64,7 @@ def _build_store(num_validators: int, genesis_time: int, anchor_slot: int = 0) -
         anchor_slot=Slot(anchor_slot),
         genesis_time=Uint64(genesis_time),
     )
-    return Store.from_anchor(state, block, validator_id=None)
+    return fork.create_store(state, block, validator_id=None)
 
 
 def _health_response(_store: Store, _fixture: "ApiEndpointTest") -> dict[str, Any]:
@@ -100,7 +100,7 @@ def _finalized_state_response(store: Store, _fixture: "ApiEndpointTest") -> dict
 
 def _fork_choice_response(store: Store, _fixture: "ApiEndpointTest") -> dict[str, Any]:
     """Fork choice tree: blocks with weights, head, checkpoints, validator count."""
-    weights = store.compute_block_weights()
+    weights = LstarSpec().compute_block_weights(store)
 
     # Only post-finalization blocks are relevant to head selection.
     nodes = [

packages/testing/src/consensus_testing/test_fixtures/fork_choice.py

@@ -11,7 +11,6 @@
 
 from pydantic import Field, model_validator
 
-from lean_spec.forks.lstar import Store
 from lean_spec.forks.lstar.containers.block import (
     Block,
     BlockBody,
@@ -38,9 +37,6 @@
 )
 from .base import BaseConsensusFixture
 
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all fixture invocations."""
-
 
 class ForkChoiceTest(BaseConsensusFixture):
     """
@@ -190,6 +186,8 @@ def make_fixture(self) -> Self:
         assert self.anchor_block is not None, "anchor block must be set before making fixture"
         assert self.max_slot is not None, "max slot must be set before making fixture"
 
+        spec = LstarSpec()
+
         # Expected anchor-init failure path.
         #
         # When anchor_valid is False, the test asserts that Store.from_anchor
@@ -202,7 +200,7 @@ def make_fixture(self) -> Self:
                 "Store.from_anchor is expected to fail before any step can run"
             )
             try:
-                Store.from_anchor(
+                spec.create_store(
                     self.anchor_state,
                     self.anchor_block,
                     validator_id=ValidatorIndex(0),
@@ -257,7 +255,7 @@ def make_fixture(self) -> Self:
         #
         # The Store is the node's local view of the chain.
         # It starts from a trusted anchor (usually genesis).
-        store = Store.from_anchor(
+        store = spec.create_store(
             self.anchor_state,
             self.anchor_block,
             validator_id=ValidatorIndex(0),
@@ -293,7 +291,7 @@ def make_fixture(self) -> Self:
                             target_interval = Interval.from_unix_time(
                                 Uint64(step.time), store.config.genesis_time
                             )
-                        store, _ = _SPEC.on_tick(
+                        store, _ = spec.on_tick(
                             store,
                             target_interval,
                             has_proposal=step.has_proposal,
@@ -326,13 +324,13 @@ def make_fixture(self) -> Self:
                         # This tick includes a block (has proposal).
                         # Always act as aggregator to ensure gossip signatures are aggregated
                         target_interval = Interval.from_slot(block.slot)
-                        store, _ = _SPEC.on_tick(
+                        store, _ = spec.on_tick(
                             store, target_interval, has_proposal=True, is_aggregator=True
                         )
 
                         # Process the block through Store.
                         # This validates, applies state transition, and updates the store's head.
-                        store = _SPEC.on_block(
+                        store = spec.on_block(
                             store,
                             signed_block,
                             scheme=LEAN_ENV_TO_SCHEMES[self.lean_env],
@@ -350,7 +348,7 @@ def make_fixture(self) -> Self:
                             step.valid,
                         )
                         step._filled_attestation = signed_attestation
-                        store = _SPEC.on_gossip_attestation(
+                        store = spec.on_gossip_attestation(
                             store,
                             signed_attestation,
                             scheme=LEAN_ENV_TO_SCHEMES[self.lean_env],
@@ -364,7 +362,7 @@ def make_fixture(self) -> Self:
                             key_manager,
                         )
                         step._filled_attestation = signed_aggregated
-                        store = _SPEC.on_gossip_aggregated_attestation(store, signed_aggregated)
+                        store = spec.on_gossip_aggregated_attestation(store, signed_aggregated)
 
                     case _:
                         raise ValueError(f"Step {i}: unknown step type {type(step).__name__}")

packages/testing/src/consensus_testing/test_fixtures/state_transition.py

@@ -17,9 +17,6 @@
 from ..test_types import AggregatedAttestationSpec, BlockSpec, StateExpectation
 from .base import BaseConsensusFixture
 
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all fixture invocations."""
-
 
 class StateTransitionTest(BaseConsensusFixture):
     """
@@ -113,6 +110,7 @@ def make_fixture(self) -> "StateTransitionTest":
         """
         actual_post_state: State | None = None
         exception_raised: Exception | None = None
+        spec = LstarSpec()
 
         # Initialize filled_blocks list that will be populated as we process blocks
         filled_blocks: list[Block] = []
@@ -140,9 +138,9 @@ def make_fixture(self) -> "StateTransitionTest":
                 if cached_state is not None:
                     state = cached_state
                 elif getattr(block_spec, "skip_slot_processing", False):
-                    state = _SPEC.process_block(state, block)
+                    state = spec.process_block(state, block)
                 else:
-                    state = _SPEC.state_transition(
+                    state = spec.state_transition(
                         state,
                         block=block,
                         valid_signatures=True,
@@ -217,7 +215,7 @@ def _build_block_from_spec(
         # Advance slots unless the spec intentionally skips slot processing.
         slot_advanced_state: State | None = None
         if not spec.skip_slot_processing:
-            slot_advanced_state = _SPEC.process_slots(state, spec.slot)
+            slot_advanced_state = LstarSpec().process_slots(state, spec.slot)
 
         # Resolve the parent root.
         # Default: latest block header from the slot-advanced state.
@@ -260,7 +258,7 @@ def _build_block_from_spec(
 
             known_block_roots = frozenset(hash_tree_root(b) for b in block_registry.values())
 
-            block, post_state, _, _ = _SPEC.build_block(
+            block, post_state, _, _ = LstarSpec().build_block(
                 state,
                 slot=spec.slot,
                 proposer_index=proposer_index,
@@ -295,8 +293,8 @@ def _build_block_from_spec(
             # The body changed, so re-run the transition to get the correct
             # post-state and state root.
             if post_state is not None:
-                post_state = _SPEC.process_slots(state, spec.slot)
-                post_state = _SPEC.process_block(post_state, block)
+                post_state = LstarSpec().process_slots(state, spec.slot)
+                post_state = LstarSpec().process_block(post_state, block)
                 block = block.model_copy(update={"state_root": hash_tree_root(post_state)})
 
         return block, post_state

packages/testing/src/consensus_testing/test_fixtures/verify_signatures.py

@@ -22,9 +22,6 @@
 from ..test_types import BlockSpec
 from .base import BaseConsensusFixture
 
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all fixture invocations."""
-
 
 class VerifySignaturesTest(BaseConsensusFixture):
     """
@@ -115,7 +112,7 @@ def make_fixture(self) -> VerifySignaturesTest:
 
         # Verify signatures
         try:
-            _SPEC.verify_signatures(signed_block, self.anchor_state.validators)
+            LstarSpec().verify_signatures(signed_block, self.anchor_state.validators)
         except AssertionError as e:
             exception_raised = e
             # If we expect an exception, this is fine

packages/testing/src/consensus_testing/test_types/block_spec.py

@@ -32,9 +32,6 @@
 from ..keys import LEAN_ENV_TO_SCHEMES, XmssKeyManager, create_dummy_signature
 from .aggregated_attestation_spec import AggregatedAttestationSpec
 
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all spec invocations."""
-
 
 class BlockSpec(CamelModel):
     """
@@ -294,6 +291,7 @@ def build_signed_block(
         Returns:
             Complete signed block with all attestation and proposer signatures.
         """
+        spec = LstarSpec()
         proposer_index = self.resolve_proposer_index(len(state.validators))
 
         # Build a genesis block registry so attestation specs can resolve labels.
@@ -308,7 +306,7 @@ def build_signed_block(
 
         # Resolve the parent root.
         # The default is the latest block header from the slot-advanced state.
-        parent_state = _SPEC.process_slots(state, self.slot)
+        parent_state = spec.process_slots(state, self.slot)
         parent_root = self.resolve_parent_root(
             block_registry,
             default_root=hash_tree_root(parent_state.latest_block_header),
@@ -364,7 +362,7 @@ def build_signed_block(
             for agg_att, proof in zip(aggregated_attestations, attestation_sigs.data, strict=True)
         }
 
-        final_block, _, _, aggregated_signatures = _SPEC.build_block(
+        final_block, _, _, aggregated_signatures = spec.build_block(
             state,
             slot=self.slot,
             proposer_index=proposer_index,
@@ -405,6 +403,7 @@ def build_signed_block_with_store(
         Returns:
             Complete signed block ready for Store processing.
         """
+        spec = LstarSpec()
         proposer_index = self.resolve_proposer_index(len(store.states[store.head].validators))
 
         # Resolve parent block.
@@ -429,7 +428,7 @@ def build_signed_block_with_store(
         # check rejects votes whose slot has not yet started locally.
         block_slot_interval = Interval.from_slot(self.slot)
         if store.time < block_slot_interval:
-            store, _ = _SPEC.on_tick(
+            store, _ = spec.on_tick(
                 store, block_slot_interval, has_proposal=True, is_aggregator=True
             )
 
@@ -442,7 +441,7 @@ def build_signed_block_with_store(
                 or (signature := sigs_for_data.get(attestation.validator_id)) is None
             ):
                 continue
-            store = _SPEC.on_gossip_attestation(
+            store = spec.on_gossip_attestation(
                 store,
                 SignedAttestation(
                     validator_id=attestation.validator_id,
@@ -454,11 +453,11 @@ def build_signed_block_with_store(
             )
 
         # Trigger Store aggregation to merge gossip signatures into known payloads.
-        aggregation_store, _ = store.aggregate()
-        merged_store = aggregation_store.accept_new_attestations()
+        aggregation_store, _ = spec.aggregate(store)
+        merged_store = spec.accept_new_attestations(aggregation_store)
 
         # Build the block through the spec's State.build_block().
-        final_block, _, _, block_proofs = _SPEC.build_block(
+        final_block, _, _, block_proofs = spec.build_block(
             parent_state,
             slot=self.slot,
             proposer_index=proposer_index,
@@ -470,9 +469,9 @@ def build_signed_block_with_store(
         # Append forced attestations that bypass the builder's MAX cap.
         # Each entry is signed and aggregated so the block carries valid proofs.
         if self.forced_attestations:
-            for spec in self.forced_attestations:
-                att_data = spec.build_attestation_data(block_registry, parent_state)
-                proof = key_manager.sign_and_aggregate(spec.validator_ids, att_data)
+            for att_spec in self.forced_attestations:
+                att_data = att_spec.build_attestation_data(block_registry, parent_state)
+                proof = key_manager.sign_and_aggregate(att_spec.validator_ids, att_data)
                 block_proofs.append(proof)
                 final_block = final_block.model_copy(
                     update={
@@ -483,7 +482,7 @@ def build_signed_block_with_store(
                                         *final_block.body.attestations.data,
                                         AggregatedAttestation(
                                             aggregation_bits=ValidatorIndices(
-                                                data=spec.validator_ids,
+                                                data=att_spec.validator_ids,
                                             ).to_aggregation_bits(),
                                             data=att_data,
                                         ),
@@ -495,8 +494,8 @@ def build_signed_block_with_store(
                 )
 
             # Recompute state root with the modified body.
-            post_state = _SPEC.process_slots(parent_state, self.slot)
-            post_state = _SPEC.process_block(post_state, final_block)
+            post_state = spec.process_slots(parent_state, self.slot)
+            post_state = spec.process_block(post_state, final_block)
             final_block = final_block.model_copy(update={"state_root": hash_tree_root(post_state)})
 
         return self._sign_block(final_block, block_proofs, proposer_index, key_manager)

packages/testing/src/consensus_testing/test_types/gossip_attestation_spec.py

@@ -12,9 +12,6 @@
 from ..keys import XmssKeyManager, create_dummy_signature
 from .utils import resolve_checkpoint
 
-_SPEC = LstarSpec()
-"""Active fork spec — stateless, safe to share across all spec invocations."""
-
 
 class GossipAttestationSpec(CamelModel):
     """
@@ -204,7 +201,7 @@ def build_signed(
             attestation_data = self.build_attestation_data(block_registry, anchor_block)
         else:
             # Honest path: use the Store's own attestation data production.
-            attestation_data = _SPEC.produce_attestation_data(store, self.slot)
+            attestation_data = LstarSpec().produce_attestation_data(store, self.slot)
 
         signature = (
             key_manager.sign_attestation_data(self.validator_id, attestation_data)