Skip to content

Commit 4115e2e

Browse files
mergify[bot]NickGaganclaude
authored
fix(inference): add _enforce_credentials=False to passthrough AsyncOpenAI client (backport #6344) (#6365)
Issue: #6343 ## What does this PR do? Fixes a regression in the `remote::passthrough` inference provider that breaks all requests (model listing, inference, background refresh) on openai >= 2.34.0. The passthrough provider hardcodes `api_key=""` when constructing its `AsyncOpenAI` client so that auth flows entirely through `default_headers`. This was valid when the project required `openai>=2.30.0`, but openai 2.34.0 added a constructor-level credentials enforcement check that raises `Missing credentials` when `api_key` is empty — before any headers are read or any request is made. The regression was introduced in v1.1.0 when [PR #6047](#6047) bumped the openai requirement from `>=2.30.0` to `>=2.41.0`, silently crossing the 2.34.0 boundary where enforcement was introduced. `_enforce_credentials=False` bypasses the constructor check without changing any auth behavior. It is available in all openai versions OGX currently requires (>=2.41.0). The bug was invisible in the test suite because all tests that exercise `_get_openai_client()` [mock `AsyncOpenAI` entirely](https://github.com/ogx-ai/ogx/blob/main/tests/unit/providers/inference/test_passthrough_forward_headers.py#L173), so the real SDK constructor — and its validation — never runs. **Impact without this fix:** - `GET /v1/models` returns `{"object":"list","data":[]}` — exception caught silently, logged only at DEBUG - `POST /v1/responses` returns HTTP 500 - Background model refresh fails silently — no models ever registered in the KV registry Verified against `quay.io/opendatahub/odh-ogx-core:latest` (openai 2.46.0). ## Test Plan Start OGX with a passthrough provider configured against any OpenAI-compatible endpoint: ```yaml providers: inference: - provider_id: anthropic-passthrough provider_type: remote::passthrough config: base_url: https://api.anthropic.com api_key: "<ANTHROPIC_API_KEY>" refresh_models: true registered_resources: models: - provider_id: anthropic-passthrough model_id: claude-haiku-4-5-20251001 provider_model_id: claude-haiku-4-5-20251001 model_type: llm ``` **Before fix — POST /v1/responses returns 500:** ```bash curl -s -X POST http://localhost:8323/v1/responses \ -H "Content-Type: application/json" \ -d '{"model":"anthropic-passthrough/claude-haiku-4-5-20251001","input":"Say hello","stream":false}' ``` Output: ``` {"detail":"An unexpected error occurred while generating the response."} HTTP 500 ``` Server log: ``` WARNING ogx.core.routing_tables.models:104 Model refresh failed error=Missing credentials. Please pass an `api_key`, `workload_identity`, `admin_api_key`, or set the `OPENAI_API_KEY` or `OPENAI_ADMIN_KEY` environment variable. ``` **After fix — POST /v1/responses returns 200:** ```bash curl -s -X POST http://localhost:8323/v1/responses \ -H "Content-Type: application/json" \ -d '{"model":"anthropic-passthrough/claude-haiku-4-5-20251001","input":"Say hello","stream":false}' ``` Output: ```json { "id": "resp_c0b3b7a8-476d-40c6-90bc-ae303378fa16", "object": "response", "model": "anthropic-passthrough/claude-haiku-4-5-20251001", "status": "completed", "output": [{"role": "assistant", "content": [{"type": "output_text", "text": "Hello!"}]}], "usage": {"input_tokens": 12, "output_tokens": 5, "total_tokens": 17} } ``` Additional details: [passthrough-debug.txt](https://github.com/user-attachments/files/30361254/passthrough-debug.txt) <hr>This is an automatic backport of pull request #6344 done by [Mergify](https://mergify.com). Signed-off-by: Nick Gagan <ngagan@redhat.com> Co-authored-by: Nick Gagan <40474241+NickGagan@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
1 parent 85c129d commit 4115e2e

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

src/ogx/providers/remote/inference/passthrough/passthrough.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,9 +87,13 @@ def _get_openai_client(self) -> AsyncOpenAI:
8787
# This avoids the "passthrough" sentinel that would send a spurious
8888
# Authorization: Bearer passthrough to every downstream, even when
8989
# forward_headers only targets non-auth headers like X-Tenant-ID.
90+
# _enforce_credentials=False is required for openai>=2.34.0, which added a
91+
# constructor-level credentials check that rejects api_key="". This parameter
92+
# is available in all versions OGX requires (>=2.41.0).
9093
return AsyncOpenAI(
9194
base_url=f"{base_url.rstrip('/')}/v1",
9295
api_key="",
96+
_enforce_credentials=False,
9397
default_headers=request_headers or None,
9498
)
9599

0 commit comments

Comments
 (0)