Skip to content

feat: add configurable multi-step Best-of-N evolution - #397

Open
Enoch-H-Kang wants to merge 1 commit into
gepa-ai:mainfrom
Enoch-H-Kang:main
Open

Enoch-H-Kang wants to merge 1 commit into
gepa-ai:mainfrom
Enoch-H-Kang:main

Conversation

@Enoch-H-Kang

Copy link
Copy Markdown

Description

This PR lets GEPA try several possible prompt/text improvements at each reflection step, pick the best-looking one, and optionally repeat that process multiple times before evaluating the final candidate.

GEPA generally produce one textual evolution per component per reflection step. With this change, users can configure:

  • bon: how many candidate textual evolutions to generate per step
  • itr: how many sequential inner evolution steps to run before evaluation
    For example, bon=3, itr=2 means GEPA generates 3 possible updates, asks the reflection LM to choose the best one, then uses that selected update as the starting point for another round of 3 updates.

What Changed

  • Added bon and itr to gepa.optimize(...).
  • Added bon and itr to optimize_anything.ReflectionConfig.
  • Implemented multi-step Best-of-N textual evolution in the modern StatelessReflectionLM path.
  • Preserved support for the standalone DSPy adapter proposal path.
  • Added metadata for inner textual-gradient iterations.
  • Added validation for invalid bon / itr values.
  • Added tests covering Best-of-N selection, chained inner iterations,
    config forwarding, and validation.

Validation

Ran the repository checks from CONTRIBUTING.md:

  • uv sync --extra dev --python 3.11
  • uv run pre-commit run --files ...
  • uv run pytest tests/
    • 564 passed, 4 skipped
  • uv run pyright
    • 0 errors, 0 warnings

@semanticdiff-com

semanticdiff-com Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review changes with  SemanticDiff

Changed Files
File Status
  src/gepa/proposer/reflective_mutation/reflective_mutation.py  60% smaller
  src/gepa/optimize_anything.py  47% smaller
  tests/test_reflection_lm.py  30% smaller
  src/gepa/proposer/reflective_mutation/reflection_lm.py  19% smaller
  src/gepa/api.py  17% smaller
  src/gepa/adapters/dspy_adapter/dspy_adapter.py  0% smaller
  tests/test_optimize.py  0% smaller

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant