perf(consensus): cache shielded bundle verification

Every shielded proof and signature is verified twice: once when the
transaction arrives over mempool gossip, and again when it arrives inside
a block. Zebra used to avoid the second pass by skipping whole-transaction
verification for block transactions already accepted into the mempool, and
removed that in #10494 as a security fix without replacing it.

The reason that bypass kept breaking is structural: it cached
`valid(tx, height, block time, spent outputs)` under a key that did not
determine that proposition. Expiry, lock time, the consensus branch id,
the Orchard soft-fork gates and the proof-size rule all move with height,
and the network upgrade even selects which Orchard circuit verifying key
applies, so a stale verdict could answer a different question than the one
being asked.

Cache the bundle verification itself instead. `verify(bundle, sighash, vk)`
is a pure function, so a hit is bit-identical to the computation it
replaces. On a hit the transaction verifier still runs end to end at the
block's height, with the block's time and the block's spent outputs; only
one `oneshot` inside `verify_sapling_bundle` or `queue_orchard_bundle` is
short-circuited.

That makes key completeness the whole of the safety argument. An entry is
keyed by the transaction's unmined ID, the sighash it was verified against,
and the shielded pool the bundle sits in:

  * the ID determines the bundle. A witnessed ID's ZIP 244 authorizing-data
    digest commits to the proofs and signatures, which the txid alone does
    not - that was CVE-2026-34377. A v4 transaction's legacy ID is the hash
    of the whole serialization, which carries the same authorizing data;
  * the sighash is named separately because it is not always a function of
    the transaction alone: a v5 or v6 sighash also commits to the amounts
    and scripts of the spent transparent outputs, and a v4 shielded sighash
    commits to the block's consensus branch id, which a v4 ID does not;
  * the pool separates the Orchard and Ironwood bundles of one v6
    transaction, which share an ID and a sighash;
  * the verifying key is committed to structurally: each Orchard circuit
    version already has its own verifier, so it now also has its own cache,
    and an entry can only be read back under the key it was written
    against. Sapling has one key pair for all of history.

Only `Ok` results are recorded. A batch error is not per-item evidence,
since `Fallback` resolves batch failures by re-verifying each item singly,
and an error out of the service need not be a verdict at all - it can
report that the batch worker shut down. Recording that as "invalid" would
make the node reject a valid block. For the same reason `Cached::poll_ready`
does not delegate to the inner service: callers poll before they call, so
delegating would surface a dead batch worker's error for an item whose
result the cache already holds. The miss path acquires inner readiness
inside `call`, which also stops hits from holding `Batch`'s semaphore
permits.

Items built without a witnessed transaction ID are verified every time.

Each cache holds 20,000 keys - several blocks of history plus a full
mempool - and reports hits, misses, inserts, evictions and size under
`zebra.consensus.cache.*`, labelled by the same `verifier` names as
`zebra.consensus.batch.duration_seconds`.

Tests: cache-key completeness for both verifiers over real mainnet bundles;
the cache's own behaviour (hit, miss, eviction, no reuse across items, no
memory of failures, cancellation, readiness) against a stub verifier; a
real Sapling bundle rejected when replayed under another branch id; and two
end-to-end tests through the transaction verifiers showing that a mempool
verification is reused by the block that mines the transaction, that the
block one height past expiry is still rejected, and that an
authorizing-data twin with an identical txid is fully re-verified and
rejected.

Author: evan-forbes

zebra-consensus/src/primitives.rs

@@ -4,6 +4,7 @@ use tokio::sync::oneshot::error::RecvError;
 
 use crate::BoxError;
 
+mod cache;
 pub mod ed25519;
 pub mod groth16;
 pub mod halo2;

zebra-consensus/src/primitives/cache.rs

@@ -0,0 +1,140 @@
+//! Tests for the shared verification cache key and the bounded store behind it.
+//!
+//! The verifiers pin their own key construction over real bundles. These cover the parts that are
+//! the same for all of them: which of the key's three components separate two entries, and the
+//! store's capacity and eviction.
+
+use zebra_chain::{
+    serialization::BytesInDisplayOrder,
+    transaction::{AuthDigest, Hash, UnminedTxId, WtxId},
+};
+
+use super::{CacheKey, ShieldedPool, VerifiedBundles};
+
+/// Returns a transaction ID with no witness, as a v1-v4 transaction has.
+fn legacy_tx_id(tag: u8) -> UnminedTxId {
+    UnminedTxId::Legacy(Hash::from_bytes_in_display_order(&[tag; 32]))
+}
+
+/// Returns a witnessed transaction ID, as a v5 or v6 transaction has.
+fn witnessed_tx_id(txid_tag: u8, auth_digest_tag: u8) -> UnminedTxId {
+    UnminedTxId::Witnessed(WtxId {
+        id: Hash::from_bytes_in_display_order(&[txid_tag; 32]),
+        auth_digest: AuthDigest::from_bytes_in_display_order(&[auth_digest_tag; 32]),
+    })
+}
+
+/// The pool separates the bundles a v6 transaction carries under one ID and one sighash.
+///
+/// The Orchard and Ironwood caches are one cache from NU6.3 onward, so this is what keeps their
+/// entries apart there.
+#[test]
+fn keys_for_the_same_transaction_differ_by_pool() {
+    let tx_id = witnessed_tx_id(1, 2);
+    let sighash = [3; 32];
+
+    let keys = [
+        CacheKey::new(tx_id, sighash, ShieldedPool::Sapling),
+        CacheKey::new(tx_id, sighash, ShieldedPool::Orchard),
+        CacheKey::new(tx_id, sighash, ShieldedPool::Ironwood),
+    ];
+
+    let unique: std::collections::HashSet<_> = keys.iter().collect();
+    assert_eq!(
+        unique.len(),
+        keys.len(),
+        "one transaction's three shielded bundles must not share a cache key"
+    );
+}
+
+/// The authorizing-data digest separates two witnessed IDs that share a txid.
+///
+/// Under ZIP 244 a v5 transaction's txid excludes its proofs and signatures, so a key that named
+/// only the txid would answer both of these with one verification. That is CVE-2026-34377.
+#[test]
+fn witnessed_keys_differ_by_authorizing_data() {
+    let sighash = [3; 32];
+
+    assert_ne!(
+        CacheKey::new(witnessed_tx_id(1, 2), sighash, ShieldedPool::Orchard),
+        CacheKey::new(witnessed_tx_id(1, 4), sighash, ShieldedPool::Orchard),
+        "the same txid with different authorizing data must not share a cache key"
+    );
+}
+
+/// The sighash separates two verifications of one transaction's bundle.
+///
+/// The sighash is not a function of the transaction alone: the amounts and scripts of the spent
+/// transparent outputs enter it, and those come from the verification context.
+#[test]
+fn keys_differ_by_sighash() {
+    let tx_id = witnessed_tx_id(1, 2);
+
+    assert_ne!(
+        CacheKey::new(tx_id, [3; 32], ShieldedPool::Sapling),
+        CacheKey::new(tx_id, [4; 32], ShieldedPool::Sapling),
+        "the sighash is an input to verification, so it must be an input to the key"
+    );
+}
+
+/// A legacy ID and a witnessed ID never collide, whatever they contain.
+#[test]
+fn legacy_and_witnessed_keys_differ() {
+    let sighash = [3; 32];
+
+    assert_ne!(
+        CacheKey::new(legacy_tx_id(1), sighash, ShieldedPool::Sapling),
+        CacheKey::new(witnessed_tx_id(1, 1), sighash, ShieldedPool::Sapling),
+        "a v4 transaction ID must not collide with a witnessed one"
+    );
+}
+
+/// Every Orchard value pool has a distinct tag.
+#[test]
+fn orchard_value_pools_map_to_distinct_tags() {
+    assert_eq!(
+        ShieldedPool::from(orchard::ValuePool::Orchard),
+        ShieldedPool::Orchard
+    );
+    assert_eq!(
+        ShieldedPool::from(orchard::ValuePool::Ironwood),
+        ShieldedPool::Ironwood
+    );
+}
+
+/// The lookup set and the eviction queue always hold the same keys.
+///
+/// They are two representations of one fact. A path that updated one without the other would
+/// either drop a key that `contains` still answers — remembering a bundle for the rest of the
+/// process — or grow the queue past the capacity it was built with.
+#[test]
+fn the_lookup_set_and_the_eviction_queue_hold_the_same_keys() {
+    let mut verified = VerifiedBundles::new(2);
+    let keys = [
+        CacheKey::new(legacy_tx_id(1), [0; 32], ShieldedPool::Sapling),
+        CacheKey::new(legacy_tx_id(2), [0; 32], ShieldedPool::Sapling),
+        CacheKey::new(legacy_tx_id(3), [0; 32], ShieldedPool::Sapling),
+    ];
+
+    for key in keys {
+        verified.insert(key);
+        assert_eq!(
+            verified.keys.len(),
+            verified.insertion_order.len(),
+            "the lookup set and the eviction queue must hold the same keys"
+        );
+        assert!(verified.keys.len() <= 2, "the capacity bounds the cache");
+    }
+    assert!(
+        !verified.contains(&keys[0]),
+        "the oldest key must be evicted"
+    );
+
+    let repeated = verified.insert(keys[2]);
+    assert!(!repeated.inserted, "a concurrent duplicate is not recorded");
+    assert_eq!(repeated.evicted, 0, "a duplicate must not evict anything");
+    assert_eq!(verified.keys.len(), verified.insertion_order.len());
+
+    verified.clear();
+    assert!(verified.keys.is_empty() && verified.insertion_order.is_empty());
+}

zebra-consensus/src/primitives/cache/tests.rs

@@ -17,7 +17,10 @@ use orchard::{
 };
 use rand::thread_rng;
 use zcash_protocol::value::ZatBalance;
-use zebra_chain::{parameters::NetworkUpgrade, transaction::SigHash};
+use zebra_chain::{
+    parameters::NetworkUpgrade,
+    transaction::{SigHash, UnminedTxId, WtxId},
+};
 
 use crate::{error::TransactionError, BoxError};
 use thiserror::Error;
@@ -26,7 +29,10 @@ use tower::Service;
 use tower_batch_control::{Batch, BatchControl, RequestWeight};
 use tower_fallback::Fallback;
 
-use super::spawn_fifo;
+use super::{
+    cache::{CacheKey, Cached, CachedItem, ShieldedPool, CACHE_CAPACITY},
+    spawn_fifo,
+};
 
 #[cfg(test)]
 mod tests;
@@ -110,16 +116,19 @@ lazy_static::lazy_static! {
 
 /// A Halo2 verification item, used as the request type of the service.
 ///
-/// An [`Item`] is key-agnostic: it carries only the bundle and sighash. The circuit era's verifying
-/// key is supplied by whichever [`Verifier`] processes the item, so an item is always validated
-/// against exactly one key and eras are never mixed within a batch.
+/// An [`Item`] is key-agnostic: the circuit era's verifying key is supplied by whichever
+/// [`Verifier`] processes the item, so an item is always validated against exactly one key and
+/// eras are never mixed within a batch. Items built by the transaction verifier also carry a
+/// cache key derived from their transaction's [`WtxId`], sighash, and bundle pool, so a bundle
+/// verified from the mempool is not verified again when the block that mines it arrives.
 #[derive(Clone, Debug)]
 pub struct Item {
     // `Arc`-wrapped so cloning an `Item` — which `tower-fallback` does eagerly for every request —
     // shares the bundle instead of deep-copying its actions and multi-KB proof. `add_bundle` only
     // needs `&Bundle`.
     bundle: Arc<orchard::bundle::Bundle<orchard::bundle::Authorized, ZatBalance>>,
     sighash: SigHash,
+    cache_key: Option<CacheKey>,
 }
 
 impl RequestWeight for Item {
@@ -130,13 +139,40 @@ impl RequestWeight for Item {
 
 impl Item {
     /// Creates a new [`Item`] from a bundle and sighash.
+    ///
+    /// Items constructed without their transaction's [`WtxId`] are verified normally but are not
+    /// cached. The transaction verifier supplies the witnessed transaction ID through a
+    /// crate-private constructor so its items can reuse successful results.
     pub fn new(
         bundle: orchard::bundle::Bundle<orchard::bundle::Authorized, ZatBalance>,
         sighash: SigHash,
     ) -> Self {
         Self {
             bundle: Arc::new(bundle),
             sighash,
+            cache_key: None,
+        }
+    }
+
+    /// Creates a cacheable item using its already-computed witnessed transaction ID.
+    ///
+    /// `wtx_id` must identify the transaction containing `bundle`. The transaction verifier
+    /// passes the ID it derived from its own request, whose caller must preserve this invariant.
+    pub(crate) fn new_with_wtx_id(
+        bundle: orchard::bundle::Bundle<orchard::bundle::Authorized, ZatBalance>,
+        sighash: SigHash,
+        wtx_id: WtxId,
+    ) -> Self {
+        let pool = ShieldedPool::from(bundle.bundle_version().value_pool());
+
+        Self {
+            bundle: Arc::new(bundle),
+            sighash,
+            cache_key: Some(CacheKey::new(
+                UnminedTxId::Witnessed(wtx_id),
+                sighash.0,
+                pool,
+            )),
         }
     }
 
@@ -156,12 +192,35 @@ impl Item {
     }
 }
 
+impl CachedItem for Item {
+    /// Returns this item's cache key, if it was constructed with a witnessed transaction ID.
+    ///
+    /// [`WtxId`] commits to the transaction's effecting and authorizing data. The sighash
+    /// additionally commits to the amounts and scripts of spent transparent outputs, which are
+    /// supplied by the verification context and are not part of the `WtxId`. The pool selects one
+    /// of the two Orchard-shaped bundles a v6 transaction can carry. The verifying key is absent
+    /// on purpose: each Orchard circuit era has its own cache, so an entry is only read back
+    /// under the key it was written against (see [`orchard_v5_verifier_for`]).
+    ///
+    /// The txid alone is insufficient because it excludes authorizing data under ZIP 244
+    /// (CVE-2026-34377). The pool is also required because both bundles in a v6 transaction share
+    /// the same [`WtxId`].
+    fn cache_key(&self) -> Option<CacheKey> {
+        self.cache_key
+    }
+}
+
 trait QueueBatchVerify {
     fn queue(&mut self, item: Item) -> Result<(), orchard::bundle::BatchError>;
 }
 
 impl QueueBatchVerify for BatchValidator<'_> {
-    fn queue(&mut self, Item { bundle, sighash }: Item) -> Result<(), orchard::bundle::BatchError> {
+    fn queue(
+        &mut self,
+        Item {
+            bundle, sighash, ..
+        }: Item,
+    ) -> Result<(), orchard::bundle::BatchError> {
         self.add_bundle(bundle.as_ref(), sighash.0)
     }
 }
@@ -221,13 +280,17 @@ impl Service<Item> for OrchardFallback {
     }
 }
 
+/// The batching-and-fallback stack for one Orchard circuit version, before caching.
+type BatchFallbackService = Fallback<Batch<Verifier, Item>, OrchardFallback>;
+
 /// The concrete type of a global Halo2 verification service.
 ///
 /// Each Orchard circuit version gets its own instance — see [`VERIFIER_PRE_NU6_2`],
-/// [`VERIFIER_NU6_2`], and [`VERIFIER_NU6_3_ONWARD`] — so that batches, fallbacks, and verifying
-/// keys are fully separated per circuit version. The Orchard verifier routing functions
-/// ([`orchard_v5_verifier_for`] / [`orchard_v6_verifier`]) return a borrow of the matching one.
-pub type VerifierService = Fallback<Batch<Verifier, Item>, OrchardFallback>;
+/// [`VERIFIER_NU6_2`], and [`VERIFIER_NU6_3_ONWARD`] — so that batches, fallbacks, verifying
+/// keys, and caches are fully separated per circuit version. The Orchard verifier routing
+/// functions ([`orchard_v5_verifier_for`] / [`orchard_v6_verifier`]) return a borrow of the
+/// matching one.
+pub type VerifierService = Cached<BatchFallbackService>;
 
 /// Builds a global Halo2 verifier that validates every item against `vk`.
 ///
@@ -236,7 +299,19 @@ pub type VerifierService = Fallback<Batch<Verifier, Item>, OrchardFallback>;
 /// passed here, so an item built by this verifier is always checked against exactly one era's key.
 /// Callers select the correct era's key by which `VERIFYING_KEY_*` they pass (see the two statics
 /// below); there is no runtime key resolution.
-fn batch_verifier(vk: &'static ItemVerifyingKey) -> VerifierService {
+///
+/// The stack is wrapped in a [`Cached`] so that a proof verified when its transaction was
+/// gossiped into the mempool does not have to be verified again when the block that mines it
+/// arrives. Because each circuit version builds its own verifier here, each also gets its own
+/// cache, which is what binds a remembered result to the `vk` it was produced under.
+/// `verifier_name` is the era's `verifier` metrics label, so each era's cache reports its own hit
+/// rate.
+fn batch_verifier(vk: &'static ItemVerifyingKey, verifier_name: &'static str) -> VerifierService {
+    Cached::new(batch_fallback_verifier(vk), CACHE_CAPACITY, verifier_name)
+}
+
+/// Builds the uncached batching-and-fallback stack for `vk`.
+fn batch_fallback_verifier(vk: &'static ItemVerifyingKey) -> BatchFallbackService {
     Fallback::new(
         Batch::new(
             Verifier::new(vk),
@@ -257,7 +332,7 @@ fn batch_verifier(vk: &'static ItemVerifyingKey) -> VerifierService {
 /// Note that making a `Service` call requires mutable access to the service, so you should call
 /// `.clone()` on the global handle to create a local, mutable handle.
 pub static VERIFIER_PRE_NU6_2: Lazy<VerifierService> =
-    Lazy::new(|| batch_verifier(&VERIFYING_KEY_PRE_NU6_2));
+    Lazy::new(|| batch_verifier(&VERIFYING_KEY_PRE_NU6_2, "halo2_pre_nu6_2"));
 
 /// Global batch verification context for **NU6.2-until-NU6.3** Halo2 Action proofs.
 ///
@@ -268,7 +343,7 @@ pub static VERIFIER_PRE_NU6_2: Lazy<VerifierService> =
 /// Note that making a `Service` call requires mutable access to the service, so you should call
 /// `.clone()` on the global handle to create a local, mutable handle.
 pub static VERIFIER_NU6_2: Lazy<VerifierService> =
-    Lazy::new(|| batch_verifier(&VERIFYING_KEY_NU6_2));
+    Lazy::new(|| batch_verifier(&VERIFYING_KEY_NU6_2, "halo2_nu6_2"));
 
 /// Global batch verification context for **NU6.3-onward** Halo2 Action proofs.
 ///
@@ -281,7 +356,7 @@ pub static VERIFIER_NU6_2: Lazy<VerifierService> =
 /// Note that making a `Service` call requires mutable access to the service, so you should call
 /// `.clone()` on the global handle to create a local, mutable handle.
 pub static VERIFIER_NU6_3_ONWARD: Lazy<VerifierService> =
-    Lazy::new(|| batch_verifier(&VERIFYING_KEY_NU6_3_ONWARD));
+    Lazy::new(|| batch_verifier(&VERIFYING_KEY_NU6_3_ONWARD, "halo2_nu6_3_onward"));
 
 /// Returns the global Halo2 verifier for the **Orchard-pool** bundle of a **v5** transaction in a
 /// block at `network_upgrade`.
@@ -337,6 +412,15 @@ pub fn orchard_v5_verifier_for(network_upgrade: NetworkUpgrade) -> &'static Veri
     }
 }
 
+/// Returns how many times `item` has reached the inner Halo2 verifier of the circuit version
+/// `network_upgrade` routes v5 Orchard bundles to.
+///
+/// Test-only. See [`Cached::inner_calls_for`].
+#[cfg(test)]
+pub(crate) fn inner_calls_for(network_upgrade: NetworkUpgrade, item: &Item) -> usize {
+    orchard_v5_verifier_for(network_upgrade).inner_calls_for(item)
+}
+
 /// Returns the global Halo2 verifier for **v6** Orchard-pool and Ironwood-pool bundles.
 ///
 /// v6 Orchard and Ironwood bundles only exist from NU6.3 onward, so they always use the NU6.3