|
508 | 508 | { |
509 | 509 | "case_id": "development-discipline-per-edit-durable-checkpoint", |
510 | 510 | "behavior": "Preserves every implementation or test edit through immediate testing, durable resume state, and delivery-mode-specific checkpointing.", |
511 | | - "prompt": "You are midway through a ticket with several small implementation and test edits. Define the exact state machine after each individual edit, including a failed immediate test, a passing immediate test, an inseparable RED-to-GREEN pair, an interruption at any transition, documentation/configuration/formatting/generated-file changes, all three delivery modes, CI running or failing after push, and the terminal full-review sequence. Explicitly assign comprehensive slow suites and say whether they may be duplicated locally. For local-only mode, explicitly cover Tiber's opt-in committed-evidence policy when commit authority is withheld. What may be edited or batched at each point?", |
| 511 | + "prompt": "You are midway through a ticket with several small implementation and test edits. Define the exact state machine after each individual edit, including a failed immediate test, a newly written test that passes unexpectedly, a passing immediate test, an inseparable RED-to-GREEN pair, an interruption at any transition, documentation/configuration/formatting/generated-file changes, all three delivery modes, an ordinary delivery hook or formatter that changes source before ticket completion, CI running or failing after push, and the terminal full-review sequence. Explicitly assign comprehensive slow suites and say whether they may be duplicated locally. For local-only mode, explicitly cover Tiber's opt-in committed-evidence policy when commit authority is withheld. What may be edited or batched at each point?", |
512 | 512 | "tags": [ |
513 | 513 | "development-discipline", |
514 | 514 | "tdd", |
|
532 | 532 | "baselineLiftThreshold": 0.1 |
533 | 533 | }, |
534 | 534 | "minPassRate": 0.67, |
535 | | - "semanticRubric": "Pass if the response requires the smallest relevant test immediately after every individual implementation-file or test-file edit and durably binds the exact snapshot to one of four states: failing, passing-awaiting-gates-or-review, committed, or pushed/delivery-mode-equivalent, with the next permitted action. On failure it prohibits commit, push, unrelated edits, and convenience changes, allowing only the next causal edit followed immediately by another test. On pass it stops implementation/test editing until bounded lightweight review, the repository fast pre-commit gate, and the authorized delivery-mode checkpoint complete: a signed commit and push where authorized, or an exact reviewed and fast-gate-passing no-commit terminal snapshot in local-only mode when commit authority is absent. It must add that Tiber's opt-in final-review policy makes a local commit repository-required when reviewed source/verification paths must be in a commit tree; if commit authority is withheld, completion blocks without a no-commit substitute or push. It permits exactly one declared inseparable RED-to-GREEN pair by recording the expected RED and causal claim, allowing only its paired implementation edit, immediately retesting, and entering the normal green checkpoint. It gives proportional tests and checkpoint rules for docs, configuration, formatting-only work, and generated or mechanically required companions while forbidding unrelated passing work from being batched. It defines local-only as the locally authorized terminal checkpoint with no remote mutation, direct-to-trunk as a normal push, and PR mode as an authorized branch push without inferring permission to open or merge; each state must resume from exact durable evidence. After push it begins the next iteration while exact-SHA CI runs, but a completed CI failure immediately preempts work through Tiber recovery. It leaves comprehensive slow suites to CI rather than duplicating them locally. Full multi-lens review begins only after all acceptance criteria and planned increments are delivered; a review finding follows the normal tested, lightweight-reviewed, signed, exactly verified, pushed checkpoint when remote delivery is authorized, or the exact local-only equivalent, before the full clean-iteration/reset protocol resumes. Fail if tests may be deferred across edits, passing changes may be convenience-batched, remotely delivered commits are unsigned or unreviewed, failed work is committed, CI success serializes every next increment, CI failure does not preempt, PR/merge authority is inferred, local-only pushes or commits without authority, policy-required committed evidence is replaced by a mutable snapshot, comprehensive gates are duplicated locally, intermediate commits require full multi-lens review, or lightweight review substitutes for terminal clean iterations.", |
| 535 | + "semanticRubric": "Pass if the response requires the smallest relevant test immediately after every individual implementation-file or test-file edit and durably binds the exact snapshot to one of four states: failing, passing-awaiting-gates-or-review, committed, or pushed/delivery-mode-equivalent, with the next permitted action. A newly written test that passes unexpectedly remains canonical failing state with an invalid-test reason and permits only the causal test rewrite plus immediate rerun; it must not introduce a fifth durable state or checkpoint that invalid test. On failure it prohibits commit, push, unrelated edits, and convenience changes, allowing only the next causal edit followed immediately by another test. On pass it stops implementation/test editing until bounded lightweight review, the repository fast pre-commit gate, and the authorized delivery-mode checkpoint complete: a signed commit and push where authorized, or an exact reviewed and fast-gate-passing no-commit terminal snapshot in local-only mode when commit authority is absent. It must add that Tiber's opt-in final-review policy makes a local commit repository-required when reviewed source/verification paths must be in a commit tree; if commit authority is withheld, completion blocks without a no-commit substitute or push. It permits exactly one declared inseparable RED-to-GREEN pair by recording the expected RED and causal claim, allowing only its paired implementation edit, immediately retesting, and entering the normal green checkpoint. It gives proportional tests and checkpoint rules for docs, configuration, formatting-only work, and generated or mechanically required companions while forbidding unrelated passing work from being batched. It defines local-only as the locally authorized terminal checkpoint with no remote mutation, direct-to-trunk as a normal push, and PR mode as an authorized branch push without inferring permission to open or merge; each state must resume from exact durable evidence. If a hook, formatter, or other ordinary delivery step changes source before ticket completion, it repeats and delivers the causal checkpoint but does not start terminal review early; terminal delta/reset applies only to an already-active terminal review or its remediation. After push it begins the next iteration while exact-SHA CI runs, but a completed CI failure immediately preempts work through Tiber recovery. It leaves comprehensive slow suites to CI rather than duplicating them locally. Full multi-lens review begins only after all acceptance criteria and planned increments are delivered; a review finding follows the normal tested, lightweight-reviewed, signed, exactly verified, pushed checkpoint when remote delivery is authorized, or the exact local-only equivalent, before the full clean-iteration/reset protocol resumes. Fail if tests may be deferred across edits, an invalid test becomes a fifth durable state or is checkpointed, passing changes may be convenience-batched, ordinary delivery mutation starts terminal review early, remotely delivered commits are unsigned or unreviewed, failed work is committed, CI success serializes every next increment, CI failure does not preempt, PR/merge authority is inferred, local-only pushes or commits without authority, policy-required committed evidence is replaced by a mutable snapshot, comprehensive gates are duplicated locally, intermediate commits require full multi-lens review, or lightweight review substitutes for terminal clean iterations.", |
536 | 536 | "hardAssertions": [], |
537 | 537 | "calibration": { |
538 | 538 | "pass": [ |
539 | | - "After each implementation or test edit, immediately run the smallest relevant test and persist the exact snapshot as failing, passing-awaiting-gates-or-review, committed, or pushed/local-equivalent plus its sole next action. Failure permits only one causal edit then another immediate test. GREEN freezes implementation/test edits until bounded lightweight review, fast pre-commit, signed commit, and the mode-authorized checkpoint. Declare and record one inseparable RED-to-GREEN pair only when necessary. Give proportional gates to docs/config/format/generated companions but never batch unrelated passing work. Local-only stays local and may use a no-commit snapshot unless Tiber policy requires committed evidence; then a local commit is required and withheld authority blocks without authorizing a push. Trunk and PR modes push only as already authorized. Start the next increment while exact-SHA CI runs, preempt immediately on failure, and reserve comprehensive suites for CI. Run full multi-lens clean iterations only after all increments are delivered; review remediation itself goes through the normal checkpoint before review resets." |
| 539 | + "After each implementation or test edit, immediately run the smallest relevant test and persist the exact snapshot as failing, passing-awaiting-gates-or-review, committed, or pushed/local-equivalent plus its sole next action. An unexpectedly passing new test stays failing with an invalid-test reason and permits only its rewrite and rerun. Failure permits only one causal edit then another immediate test. GREEN freezes implementation/test edits until bounded lightweight review, fast pre-commit, signed commit, and the mode-authorized checkpoint. Declare and record one inseparable RED-to-GREEN pair only when necessary. Give proportional gates to docs/config/format/generated companions but never batch unrelated passing work. Local-only stays local and may use a no-commit snapshot unless Tiber policy requires committed evidence; then a local commit is required and withheld authority blocks without authorizing a push. Trunk and PR modes push only as already authorized. An ordinary source-changing hook or formatter repeats the checkpoint without starting terminal review early. Start the next increment while exact-SHA CI runs, preempt immediately on failure, and reserve comprehensive suites for CI. Run full multi-lens clean iterations only after all increments are delivered; review remediation itself goes through the normal checkpoint before review resets." |
540 | 540 | ], |
541 | 541 | "fail": [ |
542 | 542 | "Make several convenient edits, run tests once, then wait for three full-review passes and every comprehensive local suite before one unsigned commit; continue working through failed CI and open or merge a PR automatically." |
|
0 commit comments