feat(config): expose dspy.RLM max_output_chars as a budget knob#1
Merged
qazbnm456 merged 1 commit intoJul 6, 2026
Merged
Conversation
dspy.RLM head+tail-truncates each REPL output (in characters, distinct from max_tokens) before it enters the planner prompt; the planner never sees the omitted middle. The kit passed only max_iterations / max_llm_calls, so this cap was stuck at dspy's 10_000 default. Add RLMConfig.max_output_chars (env RLM_MAX_OUTPUT_CHARS, default 10_000 = dspy's own, behaviour-preserving) and pass it through the same best-effort budget dict. Co-Authored-By: Claude Opus Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
dspy.RLMhead+tail-truncates each REPL output before it enters the plannerprompt — the planner never sees the omitted middle. The cap is
max_output_chars(characters, distinct from
max_tokens), but the kit only forwardsmax_iterations/max_llm_calls, so it's pinned at dspy's10000default withno config path.
This adds
RLMConfig.max_output_chars(envRLM_MAX_OUTPUT_CHARS, default10000= dspy's own → no behaviour change unless set) and passes it through theexisting best-effort budget dict in
_build_rlm.Observed in a real run
When a REPL step prints more than the cap, the planner receives only the head
and tail — here it lost 25,141 of 35,141 chars mid-output, with no way to raise
the cap from config. Langfuse trace (what the model actually received):
Output (35,141 chars):
... (25,141 characters omitted) ...
[[ ## iteration ## ]]
4/10