feat!: multi-SDK response shapes for /v1/models by leseb · Pull Request #5522 · ogx-ai/ogx

leseb · 2026-04-10T16:03:37Z

Summary

/v1/models and /v1/models/{model_id} now return SDK-appropriate response formats based on request headers
anthropic-version header → Anthropic ModelInfo shape (id, type, display_name, created_at)
x-goog-api-key header → Google Model shape (name, display_name, description)
Default (no special headers) → OpenAI shape (unchanged, backward compatible)
OpenAPI spec documents all three variants via oneOf with header parameters

Motivation

Llama Stack supports three SDK front-ends (OpenAI, Anthropic Messages, Google Interactions) but /v1/models only returned OpenAI-shaped responses. When developers point their Anthropic or Google SDK at a Llama Stack server, client.models.list() would fail or return unexpected shapes. This makes the models endpoint work natively with all three SDKs.

Changes

File	Change
`src/llama_stack_api/sdk_detection.py`	New SDK detection utility (header-based)
`src/llama_stack_api/models/models.py`	Anthropic/Google response Pydantic models
`src/llama_stack_api/models/api.py`	Extended Models protocol
`src/llama_stack/core/routing_tables/models.py`	Translation methods, extracted `_get_all_models` helper
`src/llama_stack_api/models/fastapi_routes.py`	SDK detection + dispatch in route handlers
`scripts/openapi_generator/multi_sdk.py`	New OpenAPI transform for `oneOf` response schemas

Test plan

uv run pytest tests/unit/core/ tests/unit/providers/inline/messages/ -x - all pass
uv run pre-commit run --all-files - all pass
Manual curl verification:
- curl /v1/models → OpenAI shape
- curl -H "anthropic-version: 2023-06-01" /v1/models → Anthropic shape
- curl -H "x-goog-api-key: test" /v1/models → Google shape

Breaking change

The OpenAPI spec for /v1/models changed from a single OpenAIListModelsResponse to a oneOf with three variants. Runtime behavior for existing OpenAI SDK consumers is unchanged since the default response shape is preserved.

🤖 Generated with Claude Code

leseb · 2026-04-10T16:04:19Z

@mattf @franciscojavierarceo @cdoern just draft let me know if you like the approach.

github-actions · 2026-04-10T16:04:28Z

✱ Stainless preview builds

This PR will update the llama-stack-client SDKs with the following commit message.

feat!: multi-SDK response shapes for /v1/models

⚠️

llama-stack-client-openapi studio · code

Your SDK build had at least one "warning" diagnostic.
generate ⚠️

⚠️

llama-stack-client-python studio · code

Your SDK build had at least one "warning" diagnostic.
generate ⚠️ → build ✅ → lint ✅ → test ✅
pip install https://pkg.stainless.com/s/llama-stack-client-python/f736fbdea0573e599275e1a08fec7a9ee3051e2f/ogx_client-0.8.0a1-py3-none-any.whl

⚠️

llama-stack-client-node studio · code

Your SDK build had at least one "warning" diagnostic.
generate ⚠️ → build ✅ → lint ✅ → test ✅
npm install https://pkg.stainless.com/s/llama-stack-client-node/b29614b163faa779372b3c9250ce05eee843d448/dist.tar.gz

⚠️

llama-stack-client-go studio · conflict

Your SDK build had at least one warning diagnostic.

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
If you push custom code to the preview branch, re-run this workflow to update the comment.
Last updated: 2026-05-12 13:22:58 UTC

mattf

+1 providing a client appropriate response from /v1/models

i've seen other systems respond w/ multiple formats at the same time

mattf · 2026-04-13T17:42:07Z

+    if request is None:
+        return SdkType.OPENAI
+    headers = request.headers
+    if headers.get("anthropic-version"):
+        return SdkType.ANTHROPIC
+    if headers.get("x-goog-api-key"):
+        return SdkType.GOOGLE
+    return SdkType.OPENAI


which llm likes this structure instead of -

if request: headers = request.headers if headers.get(...): return SdkType.... return SdkType.OPENAI

mattf · 2026-04-13T17:45:11Z

+        all_models = await self._get_all_models()
+        google_models = [
+            GoogleModelInfo(
+                name=f"models/{model.identifier}",


does this work across all the various google apis (gemini / vertex / others(?))?

cc @major

mergify · 2026-04-26T10:41:55Z

This pull request has merge conflicts that must be resolved before it can be merged. @leseb please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2026-05-04T20:43:41Z

This pull request has merge conflicts that must be resolved before it can be merged. @leseb please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2026-05-12T08:26:46Z

This pull request has merge conflicts that must be resolved before it can be merged. @leseb please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

The /v1/models and /v1/models/{model_id} endpoints now return SDK-appropriate response formats based on request headers. When the Anthropic SDK calls with the anthropic-version header, the response uses the Anthropic ModelInfo shape. When the Google AI SDK calls with the x-goog-api-key header, the response uses the Google Model shape. The default remains the OpenAI format for backward compatibility. The OpenAPI spec documents all three response shapes using oneOf with the SDK detection headers as optional parameters, making the spec self-documenting for consumers of any SDK. BREAKING CHANGE: the /v1/models response schema changed from a single OpenAIListModelsResponse to a oneOf with OpenAI, Anthropic, and Google variants. Existing OpenAI SDK consumers are unaffected since the default response shape is unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

…ilds Use FastAPI's standard Request injection (auto-resolved, no need for optional parameter) and add sdk_detection module to the llama-stack-api package manifest so it is included in distribution builds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

The library client expects Pydantic model return values, not JSONResponse wrappers. Only use JSONResponse for Anthropic/Google paths that need custom headers or different response shapes. The OpenAI default path returns the Pydantic model directly, matching the original behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

Replace direct Request parameter with a Depends-based SDK detection function that uses Header() injection. This allows the library client to call route functions without a Request object - the sdk parameter defaults to SdkType.OPENAI when not injected by FastAPI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

The library client's _convert_body tries to construct Depends return types from body params, which fails for non-Pydantic types like SdkType enum. Use direct Header() parameters with None defaults instead - the library client simply skips these optional params and the route falls through to OpenAI default behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

Address review feedback: - Delete sdk_detection.py which became dead code after switching to direct Header() params for SDK detection - Remove sdk_detection from pyproject.toml py-modules since it should not be part of the public API surface - Add comment clarifying that the Google models/{id} format is specific to the Gemini API and Vertex AI would need provider-aware translation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sébastien Han <seb@redhat.com>

…tions. Signed-off-by: Sébastien Han <seb@redhat.com>

Signed-off-by: Sébastien Han <seb@redhat.com>

…sts. Signed-off-by: Sébastien Han <seb@redhat.com>

…olution for multi-SDK routes. Signed-off-by: Sébastien Han <seb@redhat.com>

Signed-off-by: Sébastien Han <seb@redhat.com>

…eam merge. Signed-off-by: Sébastien Han <seb@redhat.com>

Signed-off-by: Sébastien Han <seb@redhat.com>

…eam merge. Signed-off-by: Sébastien Han <seb@redhat.com>

…arison-v2 Signed-off-by: Sébastien Han <seb@redhat.com>

Signed-off-by: Sébastien Han <seb@redhat.com>

cdoern

lgtm, this makes sense as we add other SDK support beyond openai.

meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 10, 2026

mattf reviewed Apr 13, 2026

View reviewed changes

This comment was marked as spam.

Sign in to view

mergify Bot added the needs-rebase label Apr 26, 2026

leseb force-pushed the leseb/api-spec-comparison-v2 branch from 49b99e0 to a91ff1e Compare April 26, 2026 10:45

mergify Bot removed the needs-rebase label Apr 26, 2026

mergify Bot added the needs-rebase label May 4, 2026

leseb force-pushed the leseb/api-spec-comparison-v2 branch 2 times, most recently from 2d77524 to cb1dae4 Compare May 5, 2026 10:40

mergify Bot removed the needs-rebase label May 5, 2026

leseb force-pushed the leseb/api-spec-comparison-v2 branch from cb1dae4 to 3996d8a Compare May 11, 2026 07:13

leseb marked this pull request as ready for review May 11, 2026 08:34

leseb requested review from bbrowning, cdoern, franciscojavierarceo and raghotham as code owners May 11, 2026 08:34

leseb force-pushed the leseb/api-spec-comparison-v2 branch 2 times, most recently from 06e823c to 662350a Compare May 12, 2026 07:27

mergify Bot added the needs-rebase label May 12, 2026

leseb and others added 5 commits May 12, 2026 10:29

leseb and others added 10 commits May 12, 2026 10:29

chore: Regenerate OpenAPI specs and enrich model schema field descrip…

6033f1c

…tions. Signed-off-by: Sébastien Han <seb@redhat.com>

chore: Apply OGX license header to multi-SDK OpenAPI generator.

ef1e9be

Signed-off-by: Sébastien Han <seb@redhat.com>

fix(models): Normalize Google SDK models/ prefixes in get-model reque…

5f262e6

…sts. Signed-off-by: Sébastien Han <seb@redhat.com>

fix(models): Add Anthropic pagination and robust Google SDK model res…

f955c9a

…olution for multi-SDK routes. Signed-off-by: Sébastien Han <seb@redhat.com>

chore(docs): Refresh OpenAI conformance coverage baseline after rebase.

ecebb57

Signed-off-by: Sébastien Han <seb@redhat.com>

chore(docs): Refresh OpenAI conformance coverage baseline after upstr…

3890143

…eam merge. Signed-off-by: Sébastien Han <seb@redhat.com>

test: Add integration tests for /v1/models SDK compatibility.

5c392a9

Signed-off-by: Sébastien Han <seb@redhat.com>

chore(docs): Refresh OpenAI conformance coverage baseline after upstr…

d7b151c

…eam merge. Signed-off-by: Sébastien Han <seb@redhat.com>

Merge remote-tracking branch 'upstream/main' into leseb/api-spec-comp…

939296a

…arison-v2 Signed-off-by: Sébastien Han <seb@redhat.com>

leseb force-pushed the leseb/api-spec-comparison-v2 branch from 662350a to 939296a Compare May 12, 2026 08:30

mergify Bot removed the needs-rebase label May 12, 2026

leseb added 2 commits May 12, 2026 11:17

Merge branch 'main' into leseb/api-spec-comparison-v2

a96aa2d

fix(tests): Use millisecond timeout for Google SDK models client.

e86a583

Signed-off-by: Sébastien Han <seb@redhat.com>

leseb enabled auto-merge May 12, 2026 10:00

cdoern approved these changes May 12, 2026

View reviewed changes

leseb added this pull request to the merge queue May 12, 2026

Merged via the queue into ogx-ai:main with commit bf496f4 May 12, 2026
49 checks passed

leseb deleted the leseb/api-spec-comparison-v2 branch May 12, 2026 13:20

Navigation Menu

Uh oh!

feat!: multi-SDK response shapes for /v1/models - #5522

feat!: multi-SDK response shapes for /v1/models#5522
leseb merged 17 commits into
ogx-ai:mainfrom
leseb:leseb/api-spec-comparison-v2

leseb commented Apr 10, 2026

Uh oh!

leseb commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026 •

edited

Loading

Uh oh!

mattf left a comment

Uh oh!

mattf Apr 13, 2026

Uh oh!

Uh oh!

mattf Apr 13, 2026

Uh oh!

Uh oh!

This comment was marked as spam.

This comment was marked as spam.

mergify Bot commented Apr 26, 2026

Uh oh!

mergify Bot commented May 4, 2026

Uh oh!

mergify Bot commented May 12, 2026

Uh oh!

cdoern left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

leseb commented Apr 10, 2026

Summary

Motivation

Changes

Test plan

Breaking change

Uh oh!

leseb commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✱ Stainless preview builds

Uh oh!

mattf left a comment

Choose a reason for hiding this comment

Uh oh!

mattf Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattf Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as spam.

This comment was marked as spam.

mergify Bot commented Apr 26, 2026

Uh oh!

mergify Bot commented May 4, 2026

Uh oh!

mergify Bot commented May 12, 2026

Uh oh!

cdoern left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Apr 10, 2026 •

edited

Loading