Skip to content

fix(ai-proxy): request identity encoding for translated streams#14893

Open
benissimo wants to merge 1 commit into
Kong:masterfrom
benissimo:fix/ai-gateway-sse-gzip-streaming
Open

fix(ai-proxy): request identity encoding for translated streams#14893
benissimo wants to merge 1 commit into
Kong:masterfrom
benissimo:fix/ai-gateway-sse-gzip-streaming

Conversation

@benissimo

@benissimo benissimo commented Jun 5, 2026

Copy link
Copy Markdown

Summary

Translated AI Proxy streaming paths can break when an upstream returns gzip-compressed SSE. This change requests Accept-Encoding: identity only when Kong is both:

  • handling a streaming request, and
  • not in preserve/pass-through mode.

Provider-specific Accept-Encoding behavior remains in place for non-streaming requests and preserve/pass-through streaming routes.

Why

Compressed SSE is safe when Kong can pass bytes through untouched. It is unsafe when Kong must parse and transform streaming frames because the parser needs raw SSE frames incrementally.

A concrete failure mode reproduced locally with an OpenAI-compatible upstream:

  • request is streaming
  • upstream returns gzip-compressed SSE
  • Kong must translate the upstream stream to the client format
  • downstream response is 200 + Content-Encoding: gzip + empty/unusable body

Requesting identity encoding for translated streams avoids negotiating gzip for the path that Kong must parse/transform.

Validation

Tested against a local kong/kong-gateway:3.14 Docker repro using the base ai-proxy plugin with an OpenAI-compatible upstream URL.

Before patch:

  • non-streaming request: 200, JSON body
  • streaming translated request: 200, Content-Type: text/event-stream, Content-Encoding: gzip, 0-byte body

After patch, overlaying the central normalize-request.lua change into the same container/config:

  • non-streaming request: 200, JSON body
  • streaming translated request: 200, Content-Type: text/event-stream, no Content-Encoding: gzip, readable SSE body

Example downstream SSE after patch:

event: message_start
data: {"type":"message_start",...}

event: content_block_delta
data: {"index":0,"delta":{"type":"text_delta","text":"Okay,"},...}

Tests

Updated streaming integration coverage to assert:

  • translated OpenAI streaming upstream receives Accept-Encoding: identity
  • native/preserve Bedrock streaming upstream keeps provider-specific Accept-Encoding: gzip, identity

I could not run the upstream bin/busted target in this local checkout because the Kong dev runtime is not installed (env: resty: No such file or directory). I did run git diff --check and validated the behavior in Docker as described above.

@CLAassistant

CLAassistant commented Jun 5, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@benissimo benissimo force-pushed the fix/ai-gateway-sse-gzip-streaming branch from 8dce523 to 04270da Compare June 5, 2026 19:07
@benissimo benissimo changed the title fix(ai-proxy): disable gzip for OpenAI streams fix(ai-proxy): disable gzip for translated streams Jun 5, 2026
@benissimo benissimo changed the title fix(ai-proxy): disable gzip for translated streams fix(ai-proxy): request identity encoding for translated streams Jun 5, 2026
@benissimo benissimo marked this pull request as ready for review June 5, 2026 19:12
@benissimo

Copy link
Copy Markdown
Author

Bump, it's been 2 weeks, is anyone available to review this?

@benissimo

Copy link
Copy Markdown
Author

@rolandatkong If you have a chance to review this, please let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants