feat: Generator-Evaluator dual-agent harness mode

[m18897829375]

Closes #164

Summary

This PR adds Harness Mode — a Generator-Evaluator dual-agent architecture that runs alongside the existing simple mode. In harness mode, two specialized AI agents collaborate through contract negotiation, implementation, and 4-dimension quality scoring.

Architecture

ralph.sh orchestrator
  ├── Generator (generator-prompt.md)  — builds, never judges
  └── Evaluator (evaluator-prompt.md)  — judges, never builds

Per-Story Flow

1. Contract Negotiation: Generator proposes scope → Evaluator reviews → back-and-forth until locked
2. Build: Generator reads locked contract → implements → typecheck/lint/test → commits
3. Evaluate: Evaluator tests in browser → scores on 4 dimensions → writes evaluation.json
4. Retry or Advance: Pass → next story. Fail → feedback → retry

Changes

New files (4)

File	Purpose
generator-prompt.md	Generator agent instructions — builds code, drafts contracts, never self-evaluates
evaluator-prompt.md	Evaluator/QA agent instructions — reviews contracts, tests in browser, scores, never writes code
contract.json.example	Sprint contract format reference
AGENTS.md	Architecture documentation covering both simple and harness modes

Modified (1)

File	Change
ralph.sh	Add --mode harness with contract negotiation, phase discipline, 4-dimension scoring, MCP lifecycle management, and crash recovery. All changes are purely additive — run_simple_mode() is preserved unchanged, default mode is simple.

Unchanged

CLAUDE.md, prompt.md, prd.json.example, skills/, flowchart/, .github/, LICENSE

Backward Compatibility

./ralph.sh                           # simple mode (unchanged behavior)
./ralph.sh --mode harness --tool claude  # new harness mode

Verification

• bash -n ralph.sh passes syntax check
• run_simple_mode() is unchanged
• No API keys or personal config files included
• All existing files (skills, flowchart, .github) unchanged

See RFC #164 for full design discussion.

[Keesan12]

The additive approach here is smart, and the contract step makes the mode much easier to reason about.

The question I'd keep pushing on is not just whether the evaluator scores well, but whether the artifacts make retry behavior legible later.

In these loops the expensive failures are usually:

• ambiguous retry reasons
• soft scores that do not clearly block advancement
• no durable artifact showing why a story passed versus why it was sent back

If contract.json and evaluation.json end up being stable enough for a human to audit quickly, that is where harness mode starts to feel meaningfully different from just "another dual-agent loop".

The architecture is promising; the artifact quality will probably decide how trustworthy it feels in practice.

[m18897829375]

@Keesan12 Thank you for your comments. Actually, I’ve been continuing to polish the ralph‑harness project over this period. I’ve also found that it’s quite difficult to give the evaluator appropriate prompts, and scoring the implementation results of the contract or generator is also very challenging. Although I’m not a very senior programmer, I’ve tried to strengthen the evaluator prompts by having Claude Code load some skills related to QA and code review.

[Keesan12]

That matches my experience too. The evaluator gets unstable fast when it has to both invent the rubric and apply it in the same pass.

The most reliable pattern I've seen is to shrink its job into boundary checks:

• what exact claim was the generator supposed to satisfy
• what observable artifact proves it
• is the failure blocking, partial, or informational
• what exact retry directive should be handed back

Once the evaluator is writing failure classes and retry directives instead of mostly open-ended prose scores, the loop becomes much easier to debug.

If you keep the 4-dimension scores, I would make them secondary to a fail-closed pass gate plus a compact reason code. That tends to age better once the loop starts hitting messy real-world tasks.