feat: add disable_compaction kwarg to rlm_harness#1211
feat: add disable_compaction kwarg to rlm_harness#1211
Conversation
Drops the `summarize` tool from the RLM tool registry when set, removing the agent's ability to self-compact conversation history. Useful for ablations where compaction would confound the treatment. Auto-compaction via RLM_MAX_TURNS_IN_CONTEXT is already off by default (env arg defaults to -1); this flag addresses the other compaction mechanism (the `summarize` tool). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hallerite
force-pushed
the
feat/rlm-harness-disable-compaction
branch
from
06a5eb3 to
1f9a9c9
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4120679. Configure here.
| append_to_system_prompt: str | None = None, | ||
| local_checkout: str | Path | None = None, | ||
| gh_token: str | None = None, | ||
| disable_compaction: bool = False, |
There was a problem hiding this comment.
Missing docs update for new user-facing kwarg
Low Severity
The new disable_compaction kwarg on rlm_harness() modifies user-facing functionality for RLMEnv, which is described in docs/environments.md. The project rule requires corresponding documentation updates when core user-facing functionality described in docs is added or modified. The PR description itself contains valuable context about the two compaction mechanisms (auto-compaction via RLM_MAX_TURNS_IN_CONTEXT and model-callable summarize tool) that would be useful reference material — the rule notes such information belongs in docs/faqs.md if it doesn't fit elsewhere.
Triggered by project rule: BugBot Instructions
Reviewed by Cursor Bugbot for commit 4120679. Configure here.
1 participant
Summary
Adds a
disable_compaction: bool = Falsekwarg torlm_harness(). When set, removes thesummarizetool from the harness's tool registry — the model can no longer self-compact conversation history.Motivation
In RL ablations (e.g. length-shaping studies), the
summarizetool lets the model arbitrarily collapse turns, which confounds measurements of context-length effects. We want a simple way to turn it off per-run without editingDEFAULT_RLM_TOOL_NAMESglobally.Scope
RLM has two compaction mechanisms:
RLM_MAX_TURNS_IN_CONTEXT— already off by default (env passes-1)summarizetool — hardcoded intoDEFAULT_RLM_TOOL_NAMESbefore this PR, not user-overridableThis PR addresses (2). (1) is already controllable via existing env arg.
Change
disable_compaction=False→ unchanged behavior (tool_names =["ipython", "summarize"])disable_compaction=True→tool_names = ["ipython"]Default preserves back-compat.
🤖 Generated with Claude Code
Note
Low Risk
Low risk: small, opt-in change that only filters the harness
tool_nameslist and preserves default behavior when the new flag is unset.Overview
Adds a
disable_compaction: bool = Falsekwarg torlm_harness()to optionally remove thesummarizetool from the harness tool registry.Default behavior is unchanged; when enabled, the harness passes a filtered
tool_nameslist (excludingsummarize) intoHarness(...).Reviewed by Cursor Bugbot for commit 4120679. Bugbot is set up for automated code reviews on this repo. Configure here.