Skip to content

Commit d9d0a9a

Browse files
Clarify perfect-cuboid evidence schema version (#1424)
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
1 parent abcd9a8 commit d9d0a9a

2 files changed

Lines changed: 8 additions & 3 deletions

File tree

benchmarks/datasets/conjecture-probes-v1/perfect-cuboid-scope-audit/instruction.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ be reported in any common order; the verifier compares the three aligned
1818
radicand/root pairs as an unordered set.
1919

2020
`evidence/answer.txt` must be a JSON object with exactly `schema_version`,
21-
`task_id`, `result`, and `limitations`. Use schema version `1`, the task ID and
22-
limitations from the submission contract, and the same `result` object as in
23-
`submission.json`. The file must be no larger than 2 MiB.
21+
`task_id`, `result`, and `limitations`. Use schema version `1` (the JSON string
22+
`"1"`), the task ID and limitations from the submission contract, and the same
23+
`result` object as in `submission.json`. The file must be no larger than 2 MiB.
2424

2525
This is a finite semantic-scope audit. An Euler brick is not a perfect cuboid,
2626
and finding no perfect cuboid in these twelve cases is not evidence of global

benchmarks/validation/conjecture_probes_v1/test_perfect_cuboid_scope_audit.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,11 @@
1313
TASK = ROOT / "benchmarks/datasets/conjecture-probes-v1/perfect-cuboid-scope-audit"
1414

1515

16+
def test_public_instruction_specifies_schema_version_as_json_string() -> None:
17+
instruction = (TASK / "instruction.md").read_text()
18+
assert 'Use schema version `1` (the JSON string\n`"1"`)' in instruction
19+
20+
1621
def _case(tmp_path: Path) -> tuple[Path, Path, dict]:
1722
app = tmp_path / "app"
1823
logs = tmp_path / "logs"

0 commit comments

Comments
 (0)