Validate the reassembled skill, not the bare body

[pradeeps319]

Problem

No evolved skill can ever be deployed.

evolve_skill.py validates the evolved result against evolved_body, which has no YAML frontmatter. But _check_skill_structure (constraints.py:150) requires the text to start with --- and contain name: and description:. That constraint fails unconditionally, and a failing constraint blocks deployment — so the run always ends at evolved_FAILED.md.

evolved_full is built on the immediately preceding line for exactly this purpose, then never used.

Phase 1 (skill files) is the only implemented phase, so this means the tool cannot currently produce a deployable artifact at all.

Observed

A real run improved the valset score from 0.862 to 1.0, then threw the result away:

Validating evolved skill
  ✓ size_limit: Size OK: 13216/15000 chars
  ✓ growth_limit: Growth OK: +0.0% (max +20.0%)
  ✓ non_empty: Artifact is non-empty
  ✗ skill_structure: Skill missing: YAML frontmatter (---), name field, description field
✗ Evolved skill FAILED constraints — not deploying
  Saved failed variant to output/github-code-review/evolved_FAILED.md

Running the same four constraints against the reassembled file passes all four:

PASS  size_limit - Size OK: 13856/15000 chars
PASS  growth_limit - Growth OK: +2.4% (max +20.0%)
PASS  non_empty - Artifact is non-empty
PASS  skill_structure - Skill has valid frontmatter (name + description)

Fix

One line: validate evolved_full instead of evolved_body. Existing suite passes unchanged (145 passed).

Related

The baseline check at line 121 has the same shape — it validates skill["body"], which is why every run prints a spurious skill_structure violation for the unmodified skill and then continues with "proceeding anyway". I left that alone since it only affects a warning rather than blocking deployment, but it shares the root cause and you may want to fix both together.

Separately, #159 fixes the DSPy >= 3.1 incompatibilities that cause dspy.GEPA() to be silently swapped for MIPROv2. These two are independent, but you need both before an evolution run does what the README describes end to end.

🤖 Generated with Claude Code

[TurtleMcTurtle]

Review: PR #161 — Validate reassembled skill, not bare body

This is a clean, focused, correct fix for a real bug.

The bug: evolve_skill.py validates evolved_body (bare markdown without frontmatter) against constraints that include skill_structure which requires YAML frontmatter. This means the constraint check fails unconditionally on every evolved skill — no evolved skill can ever pass validation and deploy.

The fix: Change validator.validate_all(evolved_body, ...) → validator.validate_all(evolved_full, ...) where evolved_full includes the reassembled frontmatter.

Status in other PRs:

• fix(evolution): restore GEPA skill evolution on dspy 3.2 #155 includes this same fix (plus also fixes baseline validation to use skill['raw'])
• fix: make DSPy skill evolution persist optimized instructions #153 also includes this fix
• fix: GEPA optimizer compatibility + custom OpenAI-compatible API support #167 does NOT include this fix

Suggestion: This is a one-line fix that's correct and independent. If #155 isn't merged soon, this could go in on its own. If #155 is the base PR, this is already covered there.

[pradeeps319]

Go ahead Pradeep

…

________________________________ From: TurtleZ ***@***.***> Sent: Tuesday, 04 August 2026 07:06:41 To: NousResearch/hermes-agent-self-evolution ***@***.***> Cc: Pradeep Singh ***@***.***>; Author ***@***.***> Subject: Re: [NousResearch/hermes-agent-self-evolution] Validate the reassembled skill, not the bare body (PR #161) [https://avatars.githubusercontent.com/u/61407189?s=20&v=4]TurtleMcTurtle left a comment (NousResearch/hermes-agent-self-evolution#161)<#161 (comment)> Review: PR #161<#161> — Validate reassembled skill, not bare body This is a clean, focused, correct fix for a real bug. The bug: evolve_skill.py validates evolved_body (bare markdown without frontmatter) against constraints that include skill_structure which requires YAML frontmatter. This means the constraint check fails unconditionally on every evolved skill — no evolved skill can ever pass validation and deploy. The fix: Change validator.validate_all(evolved_body, ...) → validator.validate_all(evolved_full, ...) where evolved_full includes the reassembled frontmatter. Status in other PRs: * #155<#155> includes this same fix (plus also fixes baseline validation to use skill['raw']) * #153<#153> also includes this fix * #167<#167> does NOT include this fix Suggestion: This is a one-line fix that's correct and independent. If #155<#155> isn't merged soon, this could go in on its own. If #155<#155> is the base PR, this is already covered there. — Reply to this email directly, view it on GitHub<#161?email_source=notifications&email_token=A5EN5GRUEHYDFTE5NNXE5RT5IHUXDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMJYGAYTSNJYGY2KM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-5180195864>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5EN5GUTSI6WFZLQKEUJEKT5IHUXDAVCNFSNUABGKJSXA33TNF2G64TZHMYTCNZWG42TGMRRHE5US43TOVSTWNJQGMYDQNBZG44DRILWAI>. You are receiving this because you authored the thread.Message ID: ***@***.***>