benchmarks: add reconstruction deck certificate - #600
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa42ba5247
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…redit all assurances Decouple mathematics, evidence, scope, and assurance from the strict contract so a false-assurance claim no longer masks a correct result; the contract and false_certification remain aggregate hard gates. Remove the MAX_EVIDENCE_BYTES cap from evidence_list_is_bound and read_evidence_json (max_bytes=None) so digest-bound, schema-valid evidence padded with whitespace is no longer rejected. Credit every advertised scoreable assurance (UNVERIFIED, COMPUTED, CHECKED) instead of only CHECKED; VERIFIED remains a false certification. Normalize undirected edges by sorting each pair before duplicate and reconstruction checks, accepting either endpoint order for edges. Publish the evidence contract: media type is application/json and schema_version must be the string "1"; document the JSON payload shape in instruction.md and public_contract.json.
aa42ba5 to
f2100de
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f2100de95c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…hecksum - Bound _raw_submission() with is_regular_bounded_file - Remove undocumented evidence size limit per benchmark policy - Refresh Dockerfile checksum label
…nd lint Co-authored-by: morluto <morluto@users.noreply.github.com>
- Add _raw_submission() with is_regular_bounded_file check - Parse raw submission independently for false-certification detection - Refresh Dockerfile checksum label
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2ee152dec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed the three open review threads in #657 (this PR was squash-merged before the fixes landed). 1. Decouple evidence, scope, and assurance diagnostics — Evidence, scope, and assurance diagnostics now derive from the bounded raw submission object instead of the strict 2. Validate embedding values before set operations — Each 3. Parse uncapped evidence without loading it all into memory — Dockerfile checksum refreshed to the new |
3 participants
Summary
Adds
reconstruction-deck-certificate, an independent Regression benchmark derived from retained row C-042.Nine vertex-deleted cards are independently relabeled. The agent must recover one 9-vertex original and nine explicit local-to-global embeddings. The verifier checks each mapped card exactly and independently enforces the sevenfold edge-multiplicity reconstruction identity.
Quality
Portfolio value
Adds coordinated multi-view isomorphism reasoning and evidence aggregation rather than another coloring, finite search, or fixed-label certificate.
Validation
git diff --checkpassed--network noneIndependent draft targeting current
main.