Decode streamed upstream bytes statefully so multi-byte UTF-8 characters split across chunks are not replaced with U+FFFD (#365)

[yualice97]

Fixes #365.

What

streamResponse() in proxy/stream.mjs decoded each upstream chunk independently with chunk.toString(). A multi-byte UTF-8 character that straddles a chunk boundary (any CJK character, most emoji, full-width punctuation) was emitted as two or three U+FFFD replacement characters inside the SSE JSON. That reaches Claude Code as real content — reply text, tool-call arguments, and the session transcript — and nothing downstream can repair it.

This switches the decode to Node's StringDecoder, which carries the trailing partial code point between write() calls, and flushes it with decoder.end() after the upstream stream closes.

Size

Three lines of production code plus an import. TextDecoder with { stream: true } would be equivalent; StringDecoder was chosen because the file already works with Buffers.

Evidence (red first)

test/proxy-stream-utf8-boundary.test.mjs builds one content_block_delta event on the wire and cuts the byte stream inside a CJK character, inside a 4-byte emoji, and one byte per chunk. It runs through the real streamResponse() with extSnapshot = [], so the only transform on the changed path is the decode.

• On the merge base (031d5e0): 3 of 5 fail — not ok 2/3/4, the split-character cases.
• At this head: 5 of 5 pass.
• Full suite at this head (node --test, Node 20.20): 1952 tests, 1945 pass, 0 fail, 7 skipped.

The fifth test pins the end-of-stream behaviour you asked for in #365: a stream that ends mid-character flushes the dangling bytes as U+FFFD rather than dropping them silently.

Follow-up grep (#365, point 2)

Checked the other byte paths in the package for the same class of bug:

• proxy/server.mjs collects request and response bodies with chunks.push(chunk) + Buffer.concat() and decodes once — safe.
• proxy/upstream.mjs — no per-chunk decode.
• bin/claude-via-proxy.mjs:165 does output += chunk.toString(), but only on the proxy's own stdout to match the ASCII listening on line. Harmless in practice; left alone to keep this PR minimal.

Non-Functional Requirements

• Size/complexity budget — a few lines; it is.
• Threat model — no change to inputs, trust boundaries, or what is logged. Same bytes, same decode, just with carry between chunks.
• Maintainability — no new abstraction.
• Performance/reliability — StringDecoder is a thin stateful wrapper over the same decode; no measurable cost.
• Load-bearing? — Yes: it touches the response wire path, so human review is appropriate. It does not change framing, headers, or event shape.

🤖 Generated with Claude Code