feat: forward proxy egress for upstream AI traffic#2367
Conversation
Add a spec.forwardProxy field to GatewayConfig that routes all upstream AI/LLM traffic from Gateways referencing the config through an HTTP CONNECT forward proxy. At xDS translation time the extension server resolves the proxy from the owning Gateway's GatewayConfig and wraps each upstream cluster's transport socket in Envoy's http_11_proxy transport, keeping the upstream TLS session nested inside the tunnel. This is for data planes in locked-down networks that have no direct egress to providers and must send all outbound traffic through a proxy. Fixes envoyproxy#1759 Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2367 +/- ##
==========================================
+ Coverage 84.85% 84.87% +0.01%
==========================================
Files 151 152 +1
Lines 22086 22192 +106
==========================================
+ Hits 18742 18835 +93
- Misses 2213 2220 +7
- Partials 1131 1137 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Adds table-driven coverage for resolveForwardProxyAddr (missing gateway, missing annotation, missing/empty GatewayConfig, non-Gateway parent, conflicting parents), a Get-error path, an invalid-address error path through maybeModifyCluster, and the nil transport-socket match skip. Brings forward_proxy.go over the coverage threshold. Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>
|
@gemini review |
|
/retest |
There was a problem hiding this comment.
Code Review
This pull request introduces support for routing upstream AI/LLM traffic through an HTTP CONNECT forward proxy by adding a ForwardProxy configuration field to GatewayConfig (v1alpha1 and v1beta1). The implementation wraps upstream transport sockets in Envoy's http_11_proxy transport socket. The feedback suggests improving port validation by parsing the port as a 16-bit unsigned integer to automatically reject out-of-range TCP ports, along with adding a corresponding test case.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Parse the proxy port as uint16 so ports above 65535 are rejected at config time instead of producing an invalid Envoy port. Adds a test case for the out-of-range port. Addresses review feedback. Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>
Description
Adds a
spec.forwardProxyfield toGatewayConfigthat routes all upstream AI/LLM traffic from Gateways referencing the config through an HTTP CONNECT forward proxy. This is for data planes in locked-down networks that have no direct egress to providers and must send all outbound traffic through a proxy.At xDS translation time the extension server resolves the proxy from the owning Gateway's
GatewayConfig(routeparentRefs→ Gateway →aigateway.envoyproxy.io/gateway-configannotation) and wraps each upstream cluster's transport socket in Envoy'shttp_11_proxytransport, keeping the upstream TLS session nested inside the CONNECT tunnel. InferencePool clusters are excluded, wrapping is idempotent, and it is a no-op when no proxy is configured.Config
The Gateway opts in via the existing
aigateway.envoyproxy.io/gateway-configannotation.Non-goals
Proxy authentication (
Proxy-Authorization) and TLS to the proxy itself are not supported yet — the proxy is expected to authorize the gateway by network policy (e.g. an IP allowlist).Related Issues/PRs (if applicable)
Fixes #1759