Skip to content

feat(streaming): add idleTimeout option for detecting stalled streams#959

Open
simran10meta wants to merge 1 commit intoanthropics:mainfrom
simran10meta:feat/stream-idle-timeout
Open

feat(streaming): add idleTimeout option for detecting stalled streams#959
simran10meta wants to merge 1 commit intoanthropics:mainfrom
simran10meta:feat/stream-idle-timeout

Conversation

@simran10meta
Copy link
Copy Markdown

@simran10meta simran10meta commented Mar 25, 2026

Problem

When consuming a streaming response, if the server stops sending data mid-stream without closing the connection, the consumer hangs indefinitely. The existing timeout option only applies to the initial request — there is no way to detect idle periods during an active stream.

This is particularly painful in serverless environments (Vercel, AWS Lambda, Cloudflare Workers) where function execution time is limited. A stalled stream silently consumes the entire timeout budget.

Closes #867

Solution

Adds an idleTimeout option to RequestOptions that configures a rolling idle timer on streaming responses:

  • When set, the stream throws a StreamIdleTimeoutError if no new SSE chunk arrives within the specified milliseconds
  • The timer resets on every received chunk, so it only fires when the stream goes completely silent
  • When no idleTimeout is set, behavior is completely unchanged (fully backward-compatible)
  • The AbortController is also triggered on timeout, properly cleaning up the underlying connection

Usage

const stream = await client.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: 1024,
  stream: true,
  messages: [{ role: 'user', content: 'Hello' }],
}, {
  idleTimeout: 60_000, // abort if no data for 60 seconds
});

try {
  for await (const event of stream) {
    // process events
  }
} catch (e) {
  if (e instanceof StreamIdleTimeoutError) {
    // handle stalled stream
  }
}

Design Decisions

  • Minimal surface area: One new option (idleTimeout), one new error class (StreamIdleTimeoutError). No changes to the Stream constructor or public API shape.
  • Opt-in only: Zero behavior change when idleTimeout is not set.
  • Chunk-level granularity: The timeout operates at the SSE chunk level (via iterSSEChunks), not the parsed event level. Ping events reset the timer too — correct behavior since pings indicate the connection is alive.
  • Proper cleanup: On timeout, the AbortController is aborted before the error is thrown, ensuring the underlying fetch connection is cleaned up.

Adds an `idleTimeout` option to `RequestOptions` that configures an idle
timer on streaming responses. When set, the stream will throw a
`StreamIdleTimeoutError` if no new data (SSE chunks) arrives within the
specified number of milliseconds. The timer resets on every received chunk.

This addresses a common issue in serverless environments where the server
stops sending data mid-stream without closing the connection, causing the
consumer to hang indefinitely. The existing `timeout` option only applies
to the initial request, not to idle periods during streaming.

Usage:

  const stream = await client.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 1024,
    stream: true,
    messages: [{ role: 'user', content: 'Hello' }],
  }, {
    idleTimeout: 60_000, // 60 seconds
  });

When no idleTimeout is set, behavior is unchanged (no timeout).

Closes anthropics#867
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug/proposal: infinitely hanging clients breaking bigger/complex sessions. Proposal: add Streaming Idle Timeout to Prevent Indefinite Hangs

1 participant