Skip to content

feat(MerkleTree/Inductive): reduce extractability to birthday - #365

Merged
alexanderlhicks merged 108 commits into
Verified-zkEVM:mainfrom
BoltonBailey:numina/inductive-merkle-collision-bound
May 20, 2026
Merged

feat(MerkleTree/Inductive): reduce extractability to birthday#365
alexanderlhicks merged 108 commits into
Verified-zkEVM:mainfrom
BoltonBailey:numina/inductive-merkle-collision-bound

Conversation

@BoltonBailey

@BoltonBailey BoltonBailey commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

This PR finishes the extractability proof (for the inductively structured binary Merkle trees with single-branch opening proofs).

This follows the development of the SNARGs book for the definitions, but diverges somewhat in the proof, and proves a simpler but slightly looser soundness error bound.

I made most of the initial proof with Numina's Fuse app and Claude Code, IIRC it was around 1300 lines to start. At this point I have cleaned up the proof a lot from where it was at initially, and tried to create abstractions to express theorems more succinctly. I would be open to further golfing ideas, but I would also be open to just merging this and trying to do automated refactors later.

numina-fuse Bot and others added 8 commits April 28, 2026 20:15
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…io birthday

Add `Collision.lean` and `Extractability.lean` for inductive Merkle trees, copied
from a downstream project and adapted to VCVio:

- `collisionIn` predicate (two distinct hash-oracle log entries with equal
  responses) and bridge `collisionIn_imp_logHasCollision` to the
  `OracleComp.LogHasCollision` predicate from
  `VCVio/OracleComp/QueryTracking/Birthday.lean`.
- `collision_probability_bound` is now a thin wrapper over
  `probEvent_logCollision_le_birthday_total`, taking `IsTotalQueryBound oa n`
  and concluding `n² / (2 · |α|)`.

The original `IsPerIndexQueryBound oa (fun _ => n)` hypothesis is unprovable
for `spec α := (α × α) →ₒ α` (per-index `n` over `|α|²` indices does not bound
total log length to `n`), so the consumer in `Extractability.lean` is updated
to thread a total bound through `extractability_game_IsTotalQueryBound` (still
a `sorry`, was a `sorry` under the previous per-index name as well).

Slight `n²` vs `n·(n-1)` slack relative to the textbook `C(n, 2) / |α|` form
comes from the union-bound step in the underlying birthday lemma.

Pre-existing issues in `extractability_game` (`verifyProof`/`extractedProof`
expecting `List.Vector α idx.depth` rather than `List α`) are out of scope for
this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 30, 2026

Copy link
Copy Markdown

🤖 PR Summary

Mathematical Formalization

  • Completion of the extractability theorem for inductively structured binary Merkle trees with single-branch opening proofs.
  • Security reduction of verification success against inconsistent extraction to the birthday bound of the underlying hash function, following SNARGs book definitions with a specific extraction algorithm and security game.
  • Establishment of query complexity bounds for getPutativeRoot and verifyProof in QueryBound.lean showing linear growth relative to tree depth.

Infrastructure and Library Foundations

  • Foundational lemmas added to EvalDist and OracleComp namespaces, including probEvent_mono'' and probability bounds for monadic bind operations.
  • Refined query logging infrastructure via withQueryLog redefinition and consistency proofs for nested logging to enable more succinct theorem expressions.
  • Updated LogHasCollision predicates to facilitate formal reasoning about hash collisions within query logs.

Proof Status

  • The extractability proof and all supporting lemmas are complete; no sorry or admit placeholders remain in the codebase.

Statistics

Metric Count
📝 Files Changed 9
Lines Added 1055
Lines Removed 4

Lean Declarations

✏️ **Removed:** 1 declaration(s)
  • @[reducible] def OracleComp.withQueryLog {α} (mx : OracleComp spec α) : in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
✏️ **Added:** 45 declaration(s)
  • private lemma probEvent_verifyProof_extractor_none_le_inv_card in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma chainInLog_of_extractor_internal_step_right in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma probOutput_singleHash_eq_inv_card in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma getPutativeRoot_isTotalQueryBound {s : Skeleton} in VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean
  • lemma verifyProof_isTotalQueryBound_skeleton_depth in VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean
  • private lemma chainInLog_of_mem_support_verifyProof in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma extractabilityGame_support_decompose in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma LogHasCollision.mono {log₁ log₂ : QueryLog spec} in VCVio/OracleComp/QueryTracking/Collision.lean
  • theorem populateDown_none_get_eq_none {α : Type _} {s : Skeleton} in ToMathlib/Data/IndexedBinaryTree/Basic.lean
  • private lemma probEvent_verifyProof_eq_true_eq_inv_card_of_pos_depth in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma extractabilityGame_eq_bind_verifyProof in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma extractorChildren_eq_none_of_find?_eq_none in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma verifyProof_isTotalQueryBound in VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean
  • lemma LogHasCollision.of_mem {log : QueryLog spec} in VCVio/OracleComp/QueryTracking/Collision.lean
  • private lemma singleHash_withQueryLog in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • theorem extractabilityGame_isTotalQueryBound in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma getPutativeRoot_step_withQueryLog_decompose in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • theorem logHasCollision_of_chainInLog_of_ne in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private theorem extractabilityGame_not_logHasCollision_wins_le_inv_card in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • def ChainInLog {s : Skeleton} (log : (spec α).QueryLog) (leaf root : α) : in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • theorem chainInLog_of_extractor_get_ne_none in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private theorem extractabilityGame_verified_extractor_none_le_inv_card in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma extractabilityGame_logged_prefix_map_unit_eq in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma singleHash_isTotalQueryBound (left right : α) : in VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean
  • def extractor in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • def AdversaryWinsExtractabilityGame {s : Skeleton} {AuxState : Type} : in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma chainInLog_mono {s : Skeleton} (idx : SkeletonLeafIndex s) in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • def extractabilityGame in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma probOutput_getPutativeRoot_eq_inv_card_of_pos_depth in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma probEvent_mono'' (h : ∀ x, p x → q x) : Pr[ p | mx] ≤ Pr[ q | mx] in VCVio/EvalDist/Defs/Basic.lean
  • private theorem extractabilityGame_not_logHasCollision_match in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma extractor_internal_get_eq_none_of_find?_eq_none in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma withQueryLog_bind {ι : Type} {spec : OracleSpec.{0, 0} ι} {α β : Type} in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
  • lemma probEvent_withQueryLog {ι : Type} {oSpec : OracleSpec ι} in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
  • theorem extractability [DecidableEq α] [Fintype α] [Inhabited α] in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma chainInLog_of_mem_support_getPutativeRoot in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma withQueryLog_query in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
  • def extractorChildren in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • private lemma chainInLog_of_extractor_internal_step_left in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • theorem withQueryLog_self_log_eq in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
  • @[reducible] def withQueryLog {α} (mx : OracleComp spec α) : in VCVio/OracleComp/QueryTracking/LoggingOracle.lean
  • private lemma extractor_internal_eq_of_find?_eq in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma probEvent_bind_le_of_forall_le [HasEvalSPMF m] in VCVio/EvalDist/Monad/Basic.lean
  • def Adversary.IsTwoPhaseTotalQueryBound {s : Skeleton} in VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean
  • lemma withQueryLog_pure {ι : Type} {spec : OracleSpec.{0, 0} ι} {α : Type} (x : α) : in VCVio/OracleComp/QueryTracking/LoggingOracle.lean

sorry Tracking

  • No sorrys were added, removed, or affected.

📋 **Additional Analysis**

Attribution And File Headers\n\n- Incorrect Copyright Year: The copyright headers in the new files VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean and VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean use the year 2026. Per the project instructions, this should be replaced with the current calendar year (e.g., 2024 or 2025).


📄 **Per-File Summaries**
  • ToMathlib/Data/IndexedBinaryTree/Basic.lean: This change introduces the theorem populateDown_none_get_eq_none, which proves that the populateDown operation results in a tree containing only none values when initialized with none and a branching function that preserves it. The proof is completed via induction and contains no sorry placeholders.
  • VCVio.lean: This change updates the VCVio.lean file by adding imports for the extractability and query bound properties of inductive Merkle trees, expanding the library's available cryptographic foundations.
  • VCVio/CryptoFoundations/MerkleTree/Inductive/Defs.lean: Updates the documentation for buildMerkleTreeWithHash to clarify that its functional implementation uses a bottom-up approach via the populateUp function.
  • VCVio/CryptoFoundations/MerkleTree/Inductive/Extractability.lean: This file introduces the extraction algorithm and security game for inductive Merkle trees, establishing the extractability theorem to bound the probability of verification success against an inconsistent extracted state. It defines necessary structures for adversaries and query log hash chains, providing a complete proof of the extractability property without any sorry or admit placeholders.
  • VCVio/CryptoFoundations/MerkleTree/Inductive/QueryBound.lean: This file introduces new theorems establishing formal query bounds for inductive Merkle tree primitives, specifically proving that operations like getPutativeRoot and verifyProof make a number of oracle queries bounded by the leaf or skeleton depth. The proofs are complete and do not contain any sorry or admit placeholders.
  • VCVio/EvalDist/Defs/Basic.lean: This update introduces the probEvent_mono'' lemma, which provides a simplified version of probability monotonicity by assuming a global implication between predicates instead of restricting the hypothesis to the distribution's support.
  • VCVio/EvalDist/Monad/Basic.lean: This change introduces the lemma probEvent_bind_le_of_forall_le, which establishes that an upper bound on the probability of an event within a monadic bind is maintained if it holds for every element in the support of the initial computation. The new lemma is fully proven and contains no sorry or admit placeholders.
  • VCVio/OracleComp/QueryTracking/Collision.lean: This update introduces the LogHasCollision.of_mem and LogHasCollision.mono theorems to provide a membership-based constructor and monotonicity property for the LogHasCollision predicate. These new theorems and their proofs are added without the use of any sorry or admit placeholders.
  • VCVio/OracleComp/QueryTracking/LoggingOracle.lean: This update redefines withQueryLog and introduces several new lemmas characterizing its behavior, including distribution over bind, pure, and query operations. It also adds theorems for probability preservation and a consistency proof (withQueryLog_self_log_eq) for nested query logging, with no sorry or admit placeholders used.

Last updated: 2026-05-20 16:37 UTC.

@BoltonBailey
BoltonBailey marked this pull request as ready for review May 17, 2026 02:22
@alexanderlhicks
alexanderlhicks enabled auto-merge (squash) May 20, 2026 16:34
@alexanderlhicks
alexanderlhicks merged commit 60439ce into Verified-zkEVM:main May 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants