Skip to content

feat(config): expose dspy.RLM max_output_chars as a budget knob#1

Merged
qazbnm456 merged 1 commit into
qazbnm456:mainfrom
popuku-dog:feat/configurable-max-output-chars
Jul 6, 2026
Merged

feat(config): expose dspy.RLM max_output_chars as a budget knob#1
qazbnm456 merged 1 commit into
qazbnm456:mainfrom
popuku-dog:feat/configurable-max-output-chars

Conversation

@popuku-dog

Copy link
Copy Markdown
Contributor

What & why

dspy.RLM head+tail-truncates each REPL output before it enters the planner
prompt — the planner never sees the omitted middle. The cap is max_output_chars
(characters, distinct from max_tokens), but the kit only forwards
max_iterations / max_llm_calls, so it's pinned at dspy's 10000 default with
no config path.

This adds RLMConfig.max_output_chars (env RLM_MAX_OUTPUT_CHARS, default
10000 = dspy's own → no behaviour change unless set) and passes it through the
existing best-effort budget dict in _build_rlm.

Observed in a real run

When a REPL step prints more than the cap, the planner receives only the head
and tail — here it lost 25,141 of 35,141 chars mid-output, with no way to raise
the cap from config. Langfuse trace (what the model actually received):

=== Step 3 ===
Reasoning: ...
Code:
```python
"some python code"

Output (35,141 chars):

... (25,141 characters omitted) ...

[[ ## iteration ## ]]
4/10


## Checklist

- [x] `uv run --group dev python -m pytest` is green
- [x] `uvx ruff check .` is clean
- [x] New behavior has a test
- [x] No new top-level `dspy` import in a dspy-free module (`config` / `_retry` / `sandbox` / `tools` / `trace` / `skills` / `replay` / `dataset`)
- [x] The `rlm-kit/trace/v1` trace is unchanged, or the change is additive (a new optional field) — `tests/test_contract.py` still green
- [x] The public surface stays vendor-neutral (no specific downstream project names or values)

dspy.RLM head+tail-truncates each REPL output (in characters, distinct
from max_tokens) before it enters the planner prompt; the planner never
sees the omitted middle. The kit passed only max_iterations /
max_llm_calls, so this cap was stuck at dspy's 10_000 default.

Add RLMConfig.max_output_chars (env RLM_MAX_OUTPUT_CHARS, default
10_000 = dspy's own, behaviour-preserving) and pass it through the
same best-effort budget dict.

Co-Authored-By: Claude Opus Fable 5 <noreply@anthropic.com>
@qazbnm456 qazbnm456 merged commit 7807b08 into qazbnm456:main Jul 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants