Skip to content

Commit a60d3b5

Browse files
chore: regenerate API specs and update provider compatibility matrix
1 parent 4d6c7f3 commit a60d3b5

4 files changed

Lines changed: 8 additions & 8 deletions

File tree

docs/docs/api-openai/conformance.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ This documentation is auto-generated from the OpenAI API specification compariso
2727

2828
## Integration Test Coverage
2929

30-
Overall Test Coverage Score: 44.1%
30+
Overall Test Coverage Score: 45.0%
3131

3232
| Category | Covered | Total | Score |
3333
|----------|---------|-------|-------|
@@ -36,7 +36,7 @@ Overall Test Coverage Score: 44.1%
3636
| Request Parameters | 21 | 25 | 84.0% |
3737
| Streaming Events | 16 | 53 | 30.2% |
3838
| Structured Output | 0 | 2 | 0.0% |
39-
| Tools | 2 | 16 | 12.5% |
39+
| Tools | 3 | 16 | 18.8% |
4040

4141
## Category Scores
4242

docs/docs/api-openai/provider_matrix.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ inference provider, based on integration test results.
2121
|----------|--------|---------|---------|----------|
2222
| azure | 102 | 102 | 0 | 84% |
2323
| bedrock | 27 | 27 | 0 | 22% |
24-
| ollama | 2 | 2 | 0 | 2% |
25-
| openai | 121 | 121 | 0 | 100% |
24+
| ollama | 4 | 4 | 0 | 3% |
25+
| openai | 121 | 121 | 0 | 99% |
2626
| vllm | 3 | 3 | 0 | 2% |
2727
| watsonx | 56 | 56 | 0 | 46% |
2828

@@ -34,7 +34,7 @@ Models, endpoints, and versions used during test recordings.
3434
|----------|----------|----------|--------------|
3535
| azure | gpt-4o | llama-stack-test.openai.azure.com, lls-test.openai.azure.com | openai sdk: 2.5.0 |
3636
| bedrock | openai.gpt-oss-20b | bedrock-mantle.us-east-2.api.aws, bedrock-mantle.us-west-2.api.aws | openai sdk: 2.5.0 |
37-
| ollama | gpt-oss:20b || openai sdk: 2.5.0, vllm server: 0.9.2rc2.dev136+g0b382b53a.d20250924 |
37+
| ollama | gpt-oss:20b || openai sdk: 2.5.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
3838
| openai | gpt-4o, o4-mini, text-embedding-3-small | api.openai.com | openai sdk: 2.5.0 |
3939
| vllm | Qwen/Qwen3-0.6B || openai sdk: 2.5.0, vllm server: 0.18.1rc1.dev197+g0e9358c11 |
4040
| watsonx | meta-llama/llama-3-3-70b-instruct | us-south.ml.cloud.ibm.com | openai sdk: 2.5.0 |
@@ -148,8 +148,9 @@ Models, endpoints, and versions used during test recordings.
148148

149149
| Feature | azure | bedrock | ollama | openai | vllm | watsonx |
150150
| --- | --- | --- | --- | --- | --- | --- |
151-
| reasoning basic streaming ||| ⏭️ ||||
151+
| reasoning basic streaming ||| ||||
152152
| reasoning multi turn passthrough | ⏭️ ||||| ⏭️ |
153+
| reasoning multi turn with tool call | ⏭️ | ⏭️ || ⏭️ | ⏭️ | ⏭️ |
153154
| reasoning non streaming | ⏭️ ||||| ⏭️ |
154155

155156
## Responses Access Control

src/llama_stack_api/inference/api.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717
OpenAIChatCompletionChunk,
1818
OpenAIChatCompletionChunkWithReasoning,
1919
OpenAIChatCompletionRequestWithExtraBody,
20-
OpenAIChatCompletionWithReasoning,
2120
OpenAICompletion,
2221
OpenAICompletionRequestWithExtraBody,
2322
OpenAICompletionWithInputMessages,

tests/integration/responses/test_reasoning.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ def provider_from_model(client_with_models, text_model_id):
3030

3131
def skip_if_reasoning_content_not_provided(client_with_models, text_model_id):
3232
provider_type = provider_from_model(client_with_models, text_model_id).provider_type
33-
if provider_type in ("remote::openai", "remote::azure", "remote::watsonx", "remote::vllm"):
33+
if provider_type in ("remote::openai", "remote::azure", "remote::watsonx"):
3434
pytest.skip(f"{provider_type} doesn't return reasoning content.")
3535

3636

0 commit comments

Comments
 (0)