Skip to content

benchmarks: add finite Hadwiger minor certificate - #599

Merged
morluto merged 11 commits into
mainfrom
agent/conjecture-hadwiger-minor-certificate
Aug 7, 2026
Merged

benchmarks: add finite Hadwiger minor certificate#599
morluto merged 11 commits into
mainfrom
agent/conjecture-hadwiger-minor-certificate

Conversation

@yuelgrace1810-ops

Copy link
Copy Markdown
Collaborator

Summary

Adds hadwiger-triangle-free-minor-certificate, an independent Regression benchmark derived from retained row C-008.

The agent must construct a connected triangle-free 11-vertex graph with 20 edges, a valid four-coloring, and four connected pairwise-adjacent branch sets forming a K4 minor. The verifier independently rejects every three-coloring by backtracking and validates the full minor model.

Quality

  • Primary objective: exact finite coloring/minor certification
  • Quality score: 89/100
  • Difficulty: provisional Hard
  • Alternative answers: vertex relabelings and alternative qualifying graphs are accepted
  • Shortcut audit: complete graphs, triangles, low-degree padding, copied chromatic labels, invalid colorings, disconnected/overlapping branch sets, and false global claims fail
  • Assurance boundary: one finite graph only; no global Hadwiger conclusion

Validation

  • 4 focused/adversarial tests passed, including relabeling acceptance
  • scoped Harbor prepare/check passed
  • Ruff and git diff --check passed
  • Docker Client/Server 29.6.2
  • task/verifier containers built and executed with --network none
  • direct Oracle applicable reward: 1.0
  • GitHub Linux exact-task Oracle remains required because native Harbor 0.20 on macOS rejects the no-network capability before execution

Independent draft targeting current main.

morluto commented Aug 7, 2026

Copy link
Copy Markdown
Owner

@codex

@morluto
morluto marked this pull request as ready for review August 7, 2026 09:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab0867e930

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Grace Lee Rui Yue and others added 2 commits August 7, 2026 18:08
- Canonicalize undirected edges by sorting endpoints before loop/duplicate checks
- Remove MAX_EVIDENCE_BYTES cap (max_bytes=None) for evidence parsing
- Accept all scoreable assurances (UNVERIFIED, COMPUTED, CHECKED)
- Decouple mathematics, evidence, scope, and assurance from contract
- Drop whitespace-churn commits during rebase onto origin/main
@morluto
morluto force-pushed the agent/conjecture-hadwiger-minor-certificate branch from ab0867e to 2c18c68 Compare August 7, 2026 10:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c18c68eea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

morluto and others added 4 commits August 7, 2026 18:39
…hecksum

- Bound _raw_submission() with is_regular_bounded_file
- Remove undocumented evidence size limit per benchmark policy
- Refresh Dockerfile checksum label
…ontract and lint

Co-authored-by: morluto <morluto@users.noreply.github.com>
…ublish schema version, type-faithful evidence comparison
Co-authored-by: morluto <morluto@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2dc004c12

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Co-authored-by: morluto <morluto@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 158df54d98

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@morluto

morluto commented Aug 7, 2026

Copy link
Copy Markdown
Owner

This commit addresses all 6 review threads on the Hadwiger triangle-free minor certificate verifier.

Summary of fixes

1. Decouple evidence diagnostics from strict envelope

evidence_list_is_bound() and read_evidence_json() are now called on the raw submission (raw) instead of the strictly-validated envelope (s). When load_submission() returns None due to a schema error, evidence diagnostics are no longer lost — they are still computed from the raw submission JSON. Only the protocol/aggregate diagnostic (c) uses the strict load_submission() result.

2. Require exact integer invariant

mathematics() now checks type(r.get("chromatic_number")) is int and type(r.get("minor_order")) is int before the == 4 comparison. This rejects 4.0 (float) invariants, which were previously accepted because 4.0 == 4 is True in Python.

3. Wire the Hadwiger leaf into validation planner

Added a conjecture-probes-v1 case in task_host_validation() in benchmarks/tooling/validation_plan.py. It checks for a dedicated task-specific test file (test_<task>.py) in benchmarks/validation/conjecture_probes_v1/ and selects it for focused host validation. Falls back to dataset_host_validation() when no dedicated test exists. A new test (test_conjecture_probes_v1_task_change_selects_only_leaf) confirms the wiring.

4. Validate branch entries before dict operations

In _connected_branch_sets(), the isinstance(branch, list) and integer-range checks now run before len(branch) != len(set(branch)). Previously, if branch entries were arrays (e.g. [[1, 2], [3, 4]]), set(branch) would crash with TypeError: cannot use 'list' as a set element because lists are unhashable. The check now validates that all branch entries are integers in [0, 11) before constructing the set(), so the crash is impossible.

5. Derive scope diagnostics from the raw submission

The scope diagnostic (sc) is now computed from raw (the raw submission) instead of s (the strict envelope). When load_submission() returns None, scope is still scored from the raw submission.

6. Score assurance independently of protocol

The assurance diagnostic (a) is now computed from raw (the raw submission) instead of s (the strict envelope). When the submission is schema-invalid, assurance is still scored from the raw submission.

Other

  • Updated the Dockerfile jacobian.checksum label to match the new verifier.py content (48f25d22…).
  • Verified verifier.py is syntactically valid Python.
  • All existing Hadwiger tests (4), validation plan tests (9), and host validation tests (9) pass. Lint, format, and typecheck pass.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b31ad2f03a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@morluto
morluto merged commit ab206ab into main Aug 7, 2026
41 checks passed
@morluto
morluto deleted the agent/conjecture-hadwiger-minor-certificate branch August 7, 2026 12:26
@morluto

morluto commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Follow-up PR #675 lands the four review fixes that were resolved on this PR but merged before they reached main: evidence payload schema publication (3735082210), JSON type-faithful evidence comparison (3735082215), assurance type guard (3735590129), and non-finite JSON rejection in the raw parser (3735590133).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants