fix: salvage REPL state on mid-block exceptions instead of discarding it#172
Open
mehul79 wants to merge 1 commit into
Open
fix: salvage REPL state on mid-block exceptions instead of discarding it#172mehul79 wants to merge 1 commit into
mehul79 wants to merge 1 commit into
Conversation
Every REPL-style environment (local, docker, daytona, e2b, modal, prime) executes model code as `exec(code, combined, combined)` and, on success, copies newly-assigned variables from `combined` into the persistent `_locals`/`self.locals` namespace. On an exception mid-block, none of them did this copy — any variables the code had already assigned (e.g. partial results from a loop of `llm_query`/`rlm_query` sub-calls) were silently discarded, forcing the model to redo already-completed, already-paid-for sub-calls on its next turn. This salvages `combined` into the persistent namespace in the except branch too (same logic as the success path), restores scaffold names, and appends a note to stderr so the model knows prior work survived and it can resume a partially-completed loop instead of re-running it from scratch. ipython_repl.py is unaffected — it executes cells through an actual IPython kernel, which already retains its namespace across a failed cell. Adds 3 regression tests to tests/repl/test_local_repl.py covering: prior assignments surviving a mid-block raise, scaffold names still restored after a failure, and an integration-style case where a loop that fails on its 8th of 10 sub-calls resumes from item 8 on the next block instead of re-paying for items 1-7.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
When code running in a REPL block throws an exception partway through, every environment (
local,docker,daytona,e2b,modal,prime) discards all variables that block had already assigned — even ones from expensive work like a loop ofllm_query/rlm_querysub-calls. The model has no way to know earlier sub-calls succeeded, so it usually redoes the whole loop next turn, re-paying for calls it already made.The fix
On an exception, save the variables that were assigned before the crash into the persistent namespace (the same copy the success path already does), and tell the model in stderr that this happened. Now a failed loop can resume where it left off instead of starting over.
ipython_repl.pydidn't need this — it runs cells through a real IPython kernel, which already keeps its namespace after a failed cell.Testing
ruff check/ruff format --check: clean.tests/repl/test_local_repl.py: prior variables survive a raise, scaffold names still restore after a failure, and a loop that fails on call 8 of 10 resumes from call 8 instead of redoing 1-7.