Skip to content

fix: release stream state on end to prevent memory leaks#943

Open
Gummygamer wants to merge 2 commits intoanthropics:mainfrom
Gummygamer:fix/stream-memory-leak
Open

fix: release stream state on end to prevent memory leaks#943
Gummygamer wants to merge 2 commits intoanthropics:mainfrom
Gummygamer:fix/stream-memory-leak

Conversation

@Gummygamer
Copy link
Copy Markdown

Summary

  • MessageStream and BetaMessageStream accumulate all data in messages and receivedMessages arrays for the entire lifetime of the stream object
  • When stream instances are retained after completion (e.g. in a long-running tool loop where the caller holds references to yielded streams), this causes unbounded memory growth
  • The BetaToolRunner yields a new BetaMessageStream per iteration; if the caller retains all yielded streams, each holds a copy of the full message history at that point — O(n²) total

Fix

Cache the final message in _emitFinal() (before the 'end' event fires), then after all 'end' listeners are called, clear:

  • messages (copy of all API params — grows with conversation)
  • receivedMessages (accumulates all API responses)
  • #currentMessageSnapshot, #params, #listeners

finalMessage() and finalText() continue to work via #cachedFinalMessage.

Observed impact

Claude Code sessions growing from 2GB → 20GB+ in a single session when using tool-heavy workflows. The process had to be restarted repeatedly due to OOM. The root cause was identified by extracting embedded JS from the Claude Code binary and tracing the retention chain:

  1. BetaToolRunner yields a new BetaMessageStream per iteration
  2. Claude Code pushes each yielded stream into a React ref that is never trimmed
  3. Each retained stream holds messages (copy of all params at that iteration) + receivedMessages
  4. Compaction reduces params.messages in the runner but old stream copies are unaffected

Test plan

  • Existing tests pass
  • finalMessage() resolves correctly after end
  • finalText() resolves correctly after end
  • Stream event listeners (including 'end' listeners) still fire before cleanup

When stream instances are retained after completion (e.g. Claude Code
retains all yielded BetaMessageStream objects from BetaToolRunner in a
React ref), the `messages` and `receivedMessages` arrays accumulate
the full conversation history for the entire session lifetime, causing
unbounded O(n²) memory growth.

This fix caches the final message before emitting 'end', then clears
`messages`, `receivedMessages`, `#currentMessageSnapshot`, `#params`,
and `#listeners` once the 'end' event listeners have been called.
The public `finalMessage()` and `finalText()` methods continue to work
via `#cachedFinalMessage`.

Observed impact: Claude Code sessions growing from 2GB → 20GB+ within
a single session when using tool-heavy workflows.
@Gummygamer Gummygamer requested a review from a team as a code owner March 13, 2026 19:08
Copy link
Copy Markdown

@travisbreaks travisbreaks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find on the memory leak. In long-running tool loops, retaining stream references with all their internal state (messages, params, listeners) is a real problem. The approach of caching the final message then clearing everything else is sound.

A few observations:

1. Event ordering in _emit
After the end event fires, the method clears #listeners and returns early:

if (event === 'end') {
    // ... clear state ...
    this.#listeners = {};
    return;
}

This means if any code calls _emit('abort', ...) after _emit('end'), the abort listeners will never fire (listeners dict is empty). Is there a scenario where abort could follow end? If so, the abort handler below the end block becomes unreachable.

2. Cache population timing
_emitFinal sets #cachedFinalMessage and then calls _emit('finalMessage', ...). If a finalMessage listener accesses stream.finalMessage() synchronously, it will hit #getFinalMessage() which checks the cache first. This works correctly since #cachedFinalMessage is set before the emit. Good.

However, _emitFinal is called from within the stream processing pipeline, and then _emit('end') is called later which clears receivedMessages. If there is any listener on end that accesses receivedMessages directly (not via finalMessage()), it would see an empty array. Worth documenting this behavioral change.

3. messages vs receivedMessages
Both arrays are cleared, but messages (the input params) and receivedMessages (the output) serve different purposes. A consumer might reasonably want to inspect the input messages after the stream ends (e.g., for logging the conversation turn). Clearing messages is more aggressive than strictly necessary for the leak fix. Consider keeping messages or documenting that post-end access to messages is no longer supported.

4. Missing tests
A test that verifies finalMessage() and finalText() work correctly after end fires (using the cache) would strengthen confidence in the change. Similarly, a test that the internal arrays are actually cleared (to prove the leak is fixed).

Clean, well-scoped PR. The main concern is the behavioral change for consumers who access messages or receivedMessages after the stream ends.

- Document the abort-after-end ordering guarantee: the #ended guard at
  the top of _emit prevents any event (including abort) from firing
  after end, so the abort handler below the end block is unreachable
  once the stream has completed normally.

- Document that end listeners should use finalMessage() rather than
  reading receivedMessages directly, since receivedMessages is cleared
  as part of the end-event cleanup.

- Document that messages (input params) is also cleared post-end, and
  that post-end access to it is no longer supported. Both arrays can
  accumulate O(n^2) data across tool-loop turns, so both need clearing.

- Add tests verifying:
  * finalMessage() resolves via cache after receivedMessages is cleared
  * finalText() resolves via cache after end fires
  * Both arrays are cleared after end (proving the leak is fixed)
  * The finalMessage listener receives the message before cleanup

Applies to both MessageStream and BetaMessageStream.
@Gummygamer
Copy link
Copy Markdown
Author

Thanks for the thorough review. Addressed all four points in c8a01bf:

1. Event ordering / abort after end

You're right that the abort handler at the bottom of _emit is unreachable once end has fired. The if (this.#ended) return guard at the top ensures no event can be emitted after end. The concern about a scenario where abort follows end doesn't apply here: _run routes to either the success path (which calls _emitFinal_emit('end')) or the rejection handler #handleError (which calls _emit('abort')_emit('end')), never both. Added a comment documenting this guarantee.

2. Cache population timing / receivedMessages in end listeners

Confirmed the ordering is correct: #cachedFinalMessage is set in _emitFinal before the 'end' event fires, so finalMessage() works synchronously from any listener. Added a comment warning that 'end' listeners reading receivedMessages directly will see an empty array and should use finalMessage() instead.

3. messages vs receivedMessages

Keeping messages cleared — in the tool-loop scenario, each turn appends the assistant's response snapshot to messages via _addMessageParam(messageSnapshot) at message_stop, so messages also accumulates O(n²) data across retained streams. Clearing only receivedMessages would leave the same growth pattern. Added a comment documenting that post-end access to messages is no longer supported.

4. Missing tests (added in tests/api-resources/MessageStream.test.ts):

  • finalMessage() resolves via cache after receivedMessages is cleared
  • finalText() resolves via cache after end fires
  • Both arrays are cleared after end fires (proves the leak fix)
  • finalMessage listener receives the message before cleanup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants