fix(skills): extract evolved instruction with overfit/collapse guard

[mgandal]

Problem

evolve_skill.py built the evolved skill from optimized_module.skill_text. That attribute is the SkillModule input field — the skill body is passed verbatim into the predictor on every forward pass and is never mutated by the optimizer. So the "evolved" skill was always byte-identical to the baseline.

Reproduced on a 28KB skill: input 28,431 chars → evolved 28,431 chars, constraint gate reported growth_limit: +0.0% even though MIPRO's own trial scores improved. The score delta was real; the file delta was always zero.

Root cause

GEPA/MIPRO optimize the predictor's signature instructions (TaskWithSkill docstring), not the input field. The improved procedure lives in predictor.predict.signature.instructions and was simply never read.

Fix

• Add _extract_evolved_body(): reads the optimized instruction via named_predictors(), and uses it only when it differs from the stock TaskWithSkill docstring — otherwise falls back to skill_text, so a run never overwrites a good skill with generic boilerplate or an empty diff.
• Fix the GEPA constructor: max_steps was removed in dspy>=3 and raised TypeError, silently dropping every run into the MIPROv2 fallback. Switched to max_metric_calls.

Tests

New tests/test_evolved_body_extraction.py — 3 cases:

1. unoptimized module → falls back to original body
2. rewritten instruction → extracted (differs from baseline)
3. instruction identical to baseline docstring → falls back (no phantom evolution)

Full suite: 148 passed (was 145), no regressions.

Note for maintainers

This surfaces a second, separate issue worth a follow-up: the default skill_fitness_metric (keyword overlap) can score a generic instruction candidate above a genuinely better task-specific rewrite (observed: MIPRO proposed a strong request-classifier instruction that lost to boilerplate on keyword overlap). Consider defaulting real runs to LLMJudge. Left out of this PR to keep it focused on the extraction bug.

[mgandal]

Update: empirically validated, plus a second guard added

Ran this end-to-end against a real 15KB reference skill (gpt-oss:120b, synthetic eval, 10 iterations). Two things confirmed:

1. The extraction fix works — first non-zero body delta the pipeline has ever produced on this skill. Before, skill_text always returned the original (byte-identical); now the optimized instruction is read.

2. But it surfaced a real failure mode that needed a second guard. GEPA/MIPRO optimize the signature instruction against a small synthetic eval set. For a knowledge-dense skill, the optimizer rewrites that instruction into a narrow task-specific procedure that scores well on the eval examples but throws away the bulk of the skill:

• Input: 15KB biofigure (7 icon libraries, element schema, 3 archetypes, pitfalls)
• "Evolved": 2KB — a recipe for "embed one WikiPathways SVG and export a PDF with credit", overfit to a single synthetic example
• −86% body shrink, and it passed every existing constraint — growth_limit only caps growth (+20%), never shrink.

Without a guard, _extract_evolved_body would substitute that stub and the pipeline would gut rich skills. The second commit adds a conservative SHRINK_FLOOR (0.6): if the evolved instruction is under 60% of the baseline body, fall back to the original and leave the variant in evolved_FAILED.md for human review. Genuine refinements on short procedural skills stay well above the floor.

Note for maintainers — deeper architectural question

SkillModule treats the skill body as a fixed input field and only optimizes the predictor's signature instruction (a meta-wrapper). That's well-suited to short procedural skills where the instruction is the skill, but for knowledge-dense reference skills it structurally can't "evolve the skill" — it can only overfit the wrapper. Worth considering whether reference-style skills should be a different optimization target (e.g. section-wise, or excluded). The SHRINK_FLOOR guard makes the current pipeline safe-by-default in the meantime.

Separately: GEPA still falls back to MIPRO

Even with the max_metric_calls fix, GEPA rejects the metric: it requires a 5-arg signature (gold, pred, trace, pred_name, pred_trace) and skill_fitness_metric doesn't match. The constructor fix is necessary but not sufficient to actually run GEPA (vs the MIPROv2 fallback) — a metric-signature adapter is the remaining piece. Left out of this PR to keep scope tight.

Full suite now 150 passed.

[TurtleMcTurtle]

Review: PR #146 — Overfit/collapse guard for evolved instructions

This PR introduces a genuinely valuable safety mechanism that no other PR addresses.

Key contributions:

1. _extract_evolved_body() function — Reads the optimized predictor's Signature instructions (where GEPA actually writes mutations) instead of the untouched skill_text field. This is the same core fix as fix(evolution): restore GEPA skill evolution on dspy 3.2 #155/fix: make DSPy skill evolution persist optimized instructions #153 but wrapped in a robust extraction function with fallbacks.

2. Overfit/collapse guard (SHRINK_FLOOR = 0.6) — If the evolved instruction shrinks to <60% of the baseline body size, it's rejected as an overfit collapse. This addresses a real failure mode: GEPA optimizes against a small synthetic eval set and can happily rewrite a 15KB reference skill into a 2KB task-specific recipe that scores well but loses most of the skill's content.

3. Baseline instruction detection — Falls back to original body if the optimizer left the instruction unchanged (returned the stock TaskWithSkill docstring).

4. Tests — 5 regression tests covering: unoptimized fallback, rewritten extraction, baseline identity detection, collapse rejection, and substantial rewrite acceptance.

What's missing:

• Does NOT fix the SkillModule to use Signature instructions (still uses InputField)
• Uses max_metric_calls instead of max_full_evals (minor — both exist in different DSPy versions)
• Does NOT add reflection_lm to GEPA constructor

Suggestion: The overfit/collapse guard and tests are excellent defensive code that should be cherry-picked into whichever PR becomes the base (#155 is the strongest candidate). The _extract_evolved_body() approach is more robust than directly reading predictor.predict.signature.instructions because it handles edge cases.