feat(streaming): add idleTimeout option for detecting stalled streams#959
Open
simran10meta wants to merge 1 commit intoanthropics:mainfrom
Open
feat(streaming): add idleTimeout option for detecting stalled streams#959simran10meta wants to merge 1 commit intoanthropics:mainfrom
simran10meta wants to merge 1 commit intoanthropics:mainfrom
Conversation
Adds an `idleTimeout` option to `RequestOptions` that configures an idle
timer on streaming responses. When set, the stream will throw a
`StreamIdleTimeoutError` if no new data (SSE chunks) arrives within the
specified number of milliseconds. The timer resets on every received chunk.
This addresses a common issue in serverless environments where the server
stops sending data mid-stream without closing the connection, causing the
consumer to hang indefinitely. The existing `timeout` option only applies
to the initial request, not to idle periods during streaming.
Usage:
const stream = await client.messages.create({
model: 'claude-sonnet-4-20250514',
max_tokens: 1024,
stream: true,
messages: [{ role: 'user', content: 'Hello' }],
}, {
idleTimeout: 60_000, // 60 seconds
});
When no idleTimeout is set, behavior is unchanged (no timeout).
Closes anthropics#867
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When consuming a streaming response, if the server stops sending data mid-stream without closing the connection, the consumer hangs indefinitely. The existing
timeoutoption only applies to the initial request — there is no way to detect idle periods during an active stream.This is particularly painful in serverless environments (Vercel, AWS Lambda, Cloudflare Workers) where function execution time is limited. A stalled stream silently consumes the entire timeout budget.
Closes #867
Solution
Adds an
idleTimeoutoption toRequestOptionsthat configures a rolling idle timer on streaming responses:StreamIdleTimeoutErrorif no new SSE chunk arrives within the specified millisecondsidleTimeoutis set, behavior is completely unchanged (fully backward-compatible)AbortControlleris also triggered on timeout, properly cleaning up the underlying connectionUsage
Design Decisions
idleTimeout), one new error class (StreamIdleTimeoutError). No changes to theStreamconstructor or public API shape.idleTimeoutis not set.iterSSEChunks), not the parsed event level. Ping events reset the timer too — correct behavior since pings indicate the connection is alive.AbortControlleris aborted before the error is thrown, ensuring the underlying fetch connection is cleaned up.