Issue: SkillModule couples skill_text as an input field — GEPA/MIPROv2 cannot write evolved skill text back to SKILL.md

[luckystar2026]

Title: SkillModule exposes skill_text as an input field, so neither GEPA nor MIPROv2 can write evolved skill text back to the SKILL.md file

Labels: bug, phase-1-skills

Summary

Phase 1 (evolving SKILL.md files) never actually mutates the skill text on disk. Both optimizers — GEPA (main path, dspy 3.3.0) and MIPROv2 (fallback) — improve the behavior score during evaluation, but the saved evolved_skill.md is byte-identical to baseline_skill.md.

Root cause

evolution/skills/skill_module.py defines:

class TaskWithSkill(dspy.Signature):
    """Complete a task following the provided skill instructions.
    ..."""
    skill_instructions: str = dspy.InputField(...)
    task_input: str = dspy.InputField(...)
    output: str = dspy.OutputField(...)

def __init__(self, skill_text: str):
    super().__init__()
    self.skill_text = skill_text
    self.predictor = dspy.ChainOfThought(self.TaskWithSkill)

def forward(self, task_input: str) -> dspy.Prediction:
    result = self.predictor(
        skill_instructions=self.skill_text,
        task_input=task_input,
    )
    ...

skill_text is passed as a runtime input field of the signature. GEPA (and MIPROv2) optimize the predictor's instruction — i.e. the TaskWithSkill docstring — plus few-shot demos. They never mutate self.skill_text. After compile(), optimized_module.skill_text is identical to the baseline, so:

evolved_body = optimized_module.skill_text   # == baseline body

and the reassembled SKILL.md has zero diff.

Evidence (2026-08-07, dspy 3.3.0 / gepa 1.x / DeepSeek, arxiv skill)

Run	Optimizer	Holdout behavior score	evolved vs baseline diff
output/arxiv/20260807_021031/	MIPROv2 (fallback)	0.462 → 0.495 (+7.1%)	0 lines
output/arxiv/20260807_023548/	GEPA main path (9 iterations, 68/75 rollouts)	0.431 → 0.564 (+30.7%)	0 lines

GEPA logs confirm real evolution happened (Iteration N: New subsample score … better than old … Continue to full eval and add to candidate pool), yet the final Pareto-selected program's skill_text is the original — the optimizers improved the evaluation-time instruction/few-shot combination, and that improvement cannot be written back to the skill document.

Impact

• The headline capability of Phase 1 ("optimize SKILL.md files") cannot land: skill files on disk never change.
• Constraint gates, holdout scoring, and the diff-review workflow all pass/fail on a no-op artifact.

Suggested fix (open to maintainers)

Make the skill text the optimized instruction rather than an input field, e.g.:

1. Expose skill_text as the predictor's instruction (override the signature docstring per-module instance, or use dspy.Predictor.instructions / a dynamic signature), so GEPA's reflective mutation targets it directly; or
2. Hook into optimizer.compile() output to read back the optimized instruction into self.skill_text; or
3. Use GEPA's instruction_proposer / component_selector to mutate the skill_instructions input field explicitly and read the best variant back.

A secondary observation while testing: evolve_skill.py was passing the bare body (skill["body"] / evolved_body) to validator.validate_all(), while _check_skill_structure validates the full file (expects --- frontmatter). Every run therefore reported skill_structure FAILED even for valid output. A fix for that (validate skill["raw"] / evolved_full) is addressed in an accompanying PR (constraint-validation target + GEPA compat fixes).