benchmarks: HOST_VALIDATION_DATASET_FILES is a conjecture merge hotspot

[cursor]

Problem

benchmarks/tooling/validation_plan.py keeps an explicit HOST_VALIDATION_DATASET_FILES["conjecture-probes-v1"] tuple. Each new conjecture probe PR that adds a task-owned host regression also appends one string to this same tuple.

That made concurrent PRs conflict (seen when merging #593 Hadamard against main after #597 Yang–Mills landed): both sides edited the same adjacent lines.

After #640, task-owned changes correctly select test_<task>.py via task_host_validation. The static dataset list is only needed for dataset-wide host selection, but it is still hand-maintained and incomplete.

Current skew on main

Host tests present under benchmarks/validation/conjecture_probes_v1/ include many task files (BSD, Happy Ending, Hodge, Littlewood, Navier–Stokes, Perfect Cuboid, Reconstruction Deck, Vizing, Yang–Mills, …), while HOST_VALIDATION_DATASET_FILES still only lists a subset (recently Vizing + Yang–Mills, with Hadamard pending in #593).

Dataset-wide conjecture changes therefore under-select host coverage.

Proposed direction

Prefer one of:

1. Discover benchmarks/validation/conjecture_probes_v1/test_*.py for dataset-wide selection (deterministic sorted order), or
2. Keep a generated/checked-in inventory updated by the Harbor prepare path, not hand-edited in every leaf PR.

Leaf task PRs should not need to touch validation_plan.py at all when adding a dedicated host test.

Evidence

• Merge conflict in benchmarks: construct an order-12 Hadamard certificate #593 on benchmarks/tooling/validation_plan.py (test_yang_mills_* vs test_hadamard_*).
• fix(benchmarks): repair shared CI root causes for oracle and contracts #640 already taught task-owned selection; the remaining hotspot is the static dataset tuple.

[morluto]

Closing as completed. conjecture-probes-v1 host validation is now discovered from sorted benchmarks/validation/conjecture_probes_v1/test_*.py files via _discover_conjecture_host_tests() instead of growing HOST_VALIDATION_DATASET_FILES. New leaf tests no longer require hand-editing the shared tuple described here.