CCR streaming interception never fires on the OpenAI Responses API; long Anthropic streams can leak markers via early flush #1877

Summary

CCR's streaming interception (headroom/ccr/response_handler.py) detects headroom_retrieve tool calls by byte-substring matching against Anthropic Messages SSE framing. Two consequences:

1. On the OpenAI Responses API (e.g. headroom wrap codex), interception is structurally impossible — the retrieve tool call and <<ccr:...>> markers pass through to a client that never defined the tool.
2. On the Anthropic path, long responses can leak — the buffer's early-flush heuristic yields a half-message to the client before CCR handling re-streams a continuation, producing interleaved/malformed SSE.

Observed in practice: a Codex agent received a raw <<ccr:hash,html,6.6KB>> marker in its context and attempted to "resolve" it manually. The --no-ccr help text already hints at the limitation ("also right for streaming / non-MCP clients that can't resolve an injected tool", headroom/cli/proxy.py:294-302) — this issue proposes closing the gap instead of opting out.

Related: #1876 reports corruption in the same buffered re-synthesis path (extended-thinking responses) — independent evidence that the byte-level buffered mode is fragile; the proposal below would address that class of problem structurally.

Details (file/line refs to current main)

1. Detection is Messages-API-shaped byte matching — StreamingCCRBuffer matches b'"type":"tool_use"' + the tool name in accumulated raw bytes (response_handler.py:528-529, check at :542). Responses API streams emit function_call output items (response.output_item.added events) — the pattern can never match, so detected_ccr never becomes true for Codex traffic.

2. End-of-stream detection — StreamingCCRHandler.process_stream gates on b'"stop_reason"' in accumulated (:615). That field does not exist in Responses SSE; detection_complete stays false and the stream passes through raw.

3. The provider parameter is accepted but unused in detection (StreamingCCRHandler.__init__, :570).

4. Early-flush heuristic (:629): once the buffer exceeds 10 KB without a match, all buffered chunks are yielded and the buffer cleared. Anthropic responses emit text blocks before tool_use blocks — a response with >10 KB of text followed by a retrieve call has already streamed a partial message to the client when CCR handling kicks in and re-streams a continuation (the client sees an unterminated message followed by fresh framing).

Why this matters

The design assumes markers only travel proxy→model and the retrieve call only travels model→proxy. In streaming reality there is no enforcement of that boundary: markers reach end users through (a) model echo in visible text, (b) the unintercepted tool call, and (c) the model copying compressed content into client-side tool arguments (write/bash) — where the marker persists into real files.

Proposal

A. Egress marker scrubber (defense-in-depth, independent of interception). A client-bound streaming filter with a small rolling hold-back window (markers are bounded, ≤ ~64 bytes, so the window survives chunk-boundary splits). On match:

• in tool-call arguments → substitute the original content from the CCR store (lossless restore; the hash is authoritative)
• in prose/text deltas → replace with a human-readable placeholder (e.g. [compressed content: html, 6.6KB])
• TTL-expired hash → readable placeholder

This makes "no marker ever crosses the client boundary" an enforced invariant rather than an expectation of model behavior.

B. Event-level SSE parsing per API family. Replace byte-substring detection with real event parsing behind a small adapter interface:

• Anthropic Messages: content_block_start where content_block.type == "tool_use" and name == "headroom_retrieve"; stream end via message_delta.stop_reason.
• OpenAI Responses: response.output_item.added where item.type == "function_call" and name == "headroom_retrieve"; stream end via response.completed.

Detection becomes deterministic and the unused provider parameter becomes load-bearing.

C. Event-level stream splicing to replace the 10 KB flush heuristic: stream text deltas to the client immediately; on detecting a retrieve tool call, suppress the tool_use block events and closing stop metadata, run the retrieval continuation, and splice the continuation's content block events into the still-open stream. No duplicated framing, no unbounded buffering — and re-synthesis bugs like #1876 stop being possible because there is no re-synthesis.

Happy to contribute A and B (and C if the splicing approach is acceptable) — opening this first to agree on direction.