REST stream errors

[abrown-RV]

Problem

Streaming REST endpoints (ExecuteCommandStream, ParamInfoStream, etc. commit the HTTP status line the moment the first data:
frame is flushed (headers go out as 200 OK). If the handler then fails mid-stream, there is no in-band way to tell the client it failed — the transport can only log and drop the connection. The client sees a truncated stream and cannot distinguish:

• a clean, complete stream that happened to end, from
• a stream that was cut short by a server-side error, from
• a dropped TCP connection.

gRPC does not have this problem: its terminal status is always delivered in the trailer (OK or not), so the client always learns the outcome.

Non-goal

This is not about surfacing a command's own exception outcome. A command that deliberately reports failure already does so via a normal command_response exception frame in the data channel. This proposal is only about transport/protocol failures (INTERNAL, UNAVAILABLE, CANCELLED, …) that occur after streaming has begun. Keeping these on a separate channel is the whole point — clients must not conflate "the command failed" with "the plumbing failed," because those drive different retry decisions.

Prior art

Native SSE (EventSource) has no server-sent error/terminal frame; onerror only fires for connection problems. event: error is therefore an application-level convention. Established patterns:

• Anthropic Messages SSE — named events including event: error with a JSON {"type":"error","error":{…}}, plus a clean event: message_stop.
• OpenAI streaming — data: [DONE] sentinel for clean end; mid-stream errors as a JSON object carrying an error field.
• Kubernetes watch — every event typed; failures arrive as {"type":"ERROR","object":<Status>}.
• gRPC — terminal status always present in the trailer.

Proposal

Introduce a single named SSE event, event: error, that MAY be emitted at most once, as the final event of a stream, when a transport/protocol error occurs after the response headers have been committed.

Wire shape

Data frames are unchanged (unnamed data: events carrying a command_response / param_info JSON object). The new terminal frame is the ordinary unary REST error, relocated into the stream: the HTTP status code that would have been the status line, plus the same error message body.

event: error
data: {"status":500,"error":"handler failed after partial send"}

Rules:

1. error is terminal: no further events follow it, and the server closes the stream immediately after flushing it.
2. error is emitted only on failure. A successful stream simply ends (see Open Question 1 about adding a symmetric done sentinel).
3. status is the HTTP status code the client would have received on the status line had the failure occurred before the headers were committed — i.e. ToHTTPStatus(neutralCode). This deliberately matches the lossy HTTP vocabulary the REST client already deals with everywhere else; the streamed error path is not made special with a richer code. Never a command-domain type.
4. Best-effort: if the client has already disconnected (the common CANCELLED case), the frame may not be deliverable — that is acceptable.

Error payload schema (strawman)

Deliberately not the command exception schema. It is exactly the unary REST error (HTTP status + error body), reunited in one JSON object because the streamed frame has no status line to carry the number:

stream_error:
  type: object
  additionalProperties: false
  required: [status]
  properties:
    status:
      type: integer
      description: >
        HTTP status code the client would have received on the status line.
        Same lossy HTTP vocabulary used by every other REST error.
      example: 500
    error:
      type: string
      description: >
        Human-readable diagnostic, same shape and dev-mode redaction as the
        unary REST error body.

OpenAPI sketch

Extend the streaming 200 response so the event stream documents two event kinds. Illustrative only — adapt to the interface repo's conventions:

responses:
  "200":
    description: |
      Server-Sent Events. Data events carry command responses. If the stream
      fails after it has begun, a single terminal `error` event is emitted.
    content:
      text/event-stream:
        schema:
          type: string
        example: |
          data: <stringified {"response":{"string_value":"tick 1"}}>
          data: <stringified {"response":{"string_value":"tick 2"}}>
          event: error
          data: <stringified {"status":500,"error":"handler failed"}>
        x-sse-event-schema:
          oneOf:
            - $ref: "../../schemata/device.json#/$defs/command_response"
            - $ref: "../../schemata/device.json#/$defs/stream_error"

Semantics for implementers

• Before the first data: frame is flushed, errors are reported the normal way (real HTTP status via the status line) — no error event, since the headers are not yet committed.
• After the first data: frame, a handler error is reported as the terminal error event on a best-effort basis, then the connection is closed.
• Applies uniformly to every SSE endpoint so client stream-handling is shared.

Open questions

1. Symmetric done sentinel? gRPC always terminates with a status, letting clients distinguish clean completion from a dropped connection. We could add an event: done for the success path. Pro: unambiguous completion + client knows when to hang up. Con: changes the happy path and adds chatter; clients have tolerated its absence so far. Recommend error-only as the MVP and done as a follow-up if a client needs positive completion.
2. Redaction. error follows the same dev-mode vs prod redaction as unary error bodies (that is the point of reusing the unary shape).
3. HTTP status vs neutral code. Settled in favour of the HTTP status code for consistency: the REST client already tolerates the lossy HTTP mapping on every other endpoint, so the streamed error should not be special-cased with a finer-grained neutral code. Revisit only if a REST client emerges that needs the granularity (it would need it everywhere, not just here).
4. Event id / retry. Do we want id: / retry: for reconnection, or is command/param-info streaming inherently non-resumable? (Leaning non-resumable; out of scope here.)

What this unblocks in the Go SDK

Today streamExecuteCommand / streamParamInfo can only log on a post-header error:

if stream.sent == 0 {
    t.writeHTTPStatusResult(w, result) // headers not committed yet: real status
} else {
    logger.Error("command stream handler error after partial send", ...) // in-band signal lost
}

With this event defined, the else branch would additionally flush a single event: error frame carrying the stream_error payload — the SSE analogue of gRPC's terminal status.

[mejohnnaylor]

This is senior level writing, Andrew:

• clear understanding of SSE & HTTP lifecycle semantics
• disciplined separation of domain and transport errors
• awareness of client retry consequences
• good use of prior art
• not going mad with the proposed wire format
• attention to redaction, backwards compatibility and implementation timing.

what test cases do you propose for this?