Skip to content

Commit 112300c

Browse files
committed
pre-commit run -a
1 parent 966b7a5 commit 112300c

2 files changed

Lines changed: 20 additions & 21 deletions

File tree

docs/docs/api-openai/provider_matrix.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,13 @@ inference provider, based on integration test results.
1919

2020
| Provider | Tested | Passing | Failing | Coverage |
2121
|----------|--------|---------|---------|----------|
22-
| azure | 111 | 111 | 0 | 81% |
22+
| azure | 111 | 111 | 0 | 82% |
2323
| bedrock | 27 | 27 | 0 | 20% |
24-
| ollama | 4 | 4 | 0 | 3% |
25-
| openai | 136 | 136 | 0 | 99% |
26-
| vertexai | 70 | 70 | 0 | 51% |
27-
| vllm | 4 | 4 | 0 | 3% |
28-
| watsonx | 61 | 61 | 0 | 44% |
24+
| ollama | 3 | 3 | 0 | 2% |
25+
| openai | 136 | 136 | 0 | 100% |
26+
| vertexai | 70 | 70 | 0 | 52% |
27+
| vllm | 3 | 3 | 0 | 2% |
28+
| watsonx | 61 | 61 | 0 | 45% |
2929

3030
## Provider Details
3131

@@ -35,7 +35,7 @@ Models, endpoints, and versions used during test recordings.
3535
|----------|----------|----------|--------------|
3636
| azure | gpt-4o | llama-stack-test.openai.azure.com, lls-test.openai.azure.com, ogx-test.openai.azure.com | openai sdk: 2.43.0 |
3737
| bedrock | openai.gpt-oss-20b-1:0 | bedrock-runtime.us-west-2.amazonaws.com | openai sdk: 2.30.0 |
38-
| ollama | deepseek-r1:1.5b, gpt-oss:20b || openai sdk: 2.43.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
38+
| ollama | deepseek-r1:1.5b || openai sdk: 2.43.0 |
3939
| openai | gpt-4o, o4-mini, text-embedding-3-small | api.openai.com | openai sdk: 2.43.0 |
4040
| vertexai | publishers/google/models/gemini-2.0-flash || openai sdk: 2.5.0, provider: vertexai |
4141
| vllm | Qwen/Qwen3-0.6B || openai sdk: 2.5.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
@@ -167,7 +167,6 @@ Models, endpoints, and versions used during test recordings.
167167
| --- | --- | --- | --- | --- | --- | --- | --- |
168168
| reasoning basic streaming ||||||||
169169
| reasoning multi turn passthrough | ⏭️ |||| ⏭️ |||
170-
| reasoning multi turn with tool call | ⏭️ | ⏭️ || ⏭️ | ⏭️ || ⏭️ |
171170
| reasoning no summary without request | ⏭️ | ⏭️ | ⏭️ || ⏭️ | ⏭️ ||
172171
| reasoning non streaming | ⏭️ |||| ⏭️ |||
173172
| reasoning summary event ordering | ⏭️ | ⏭️ | ⏭️ || ⏭️ | ⏭️ ||

tests/integration/TARGET_MODELS.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,15 @@ These jobs come from the `default` section of `ci_matrix.json`. They all run in
1414
| `bedrock` | `bedrock` | library client only; 3 roots |
1515
| `base` | `ollama-postgres` | server client only; Postgres store |
1616
| `vision` | `ollama-vision` | `test_vision_inference.py` only |
17-
| `responses` | `gpt` | `responses` only; Responses coverage: 136/137 (99%) |
18-
| `responses` | `azure` | `responses` only; Responses coverage: 111/137 (81%) |
19-
| `gpt-reasoning` | `gpt-reasoning` | 2 roots; Responses coverage: 136/137 (99%) |
20-
| `responses` | `watsonx` | `responses` only; Responses coverage: 61/137 (45%) |
21-
| `responses` | `vertexai` | `responses` only; Responses coverage: 70/137 (51%) |
22-
| `bedrock-responses` | `bedrock` | 6 roots; Responses coverage: 27/137 (20%) |
17+
| `responses` | `gpt` | `responses` only; Responses coverage: 136/136 (100%) |
18+
| `responses` | `azure` | `responses` only; Responses coverage: 111/136 (82%) |
19+
| `gpt-reasoning` | `gpt-reasoning` | 2 roots; Responses coverage: 136/136 (100%) |
20+
| `responses` | `watsonx` | `responses` only; Responses coverage: 61/136 (45%) |
21+
| `responses` | `vertexai` | `responses` only; Responses coverage: 70/136 (51%) |
22+
| `bedrock-responses` | `bedrock` | 6 roots; Responses coverage: 27/136 (20%) |
2323
| `base-vllm-subset` | `vllm` | `inference` only |
24-
| `vllm-reasoning` | `vllm` | `test_reasoning.py` only; Responses coverage: 4/137 (3%) |
25-
| `ollama-reasoning` | `ollama-reasoning` | 5 roots; Responses coverage: 4/137 (3%) |
24+
| `vllm-reasoning` | `vllm` | `test_reasoning.py` only; Responses coverage: 3/136 (2%) |
25+
| `ollama-reasoning` | `ollama-reasoning` | 5 roots; Responses coverage: 3/136 (2%) |
2626
| `messages` | `ollama` | `messages` only |
2727
| `messages-openai` | `gpt` | `messages` only |
2828
| `interactions` | `gemini` | `interactions` only |
@@ -75,12 +75,12 @@ This section is derived from the same replay recordings used to generate `docs/d
7575

7676
| Provider | Tested | Passing | Coverage |
7777
|----------|--------|---------|----------|
78-
| OpenAI | 136 | 136 | 99% |
79-
| Azure | 111 | 111 | 81% |
78+
| OpenAI | 136 | 136 | 100% |
79+
| Azure | 111 | 111 | 82% |
8080
| Vertex AI | 70 | 70 | 51% |
8181
| WatsonX | 61 | 61 | 45% |
8282
| Bedrock | 27 | 27 | 20% |
83-
| Ollama | 4 | 4 | 3% |
84-
| vLLM | 4 | 4 | 3% |
83+
| Ollama | 3 | 3 | 2% |
84+
| vLLM | 3 | 3 | 2% |
8585

86-
Total Responses features counted: 137.
86+
Total Responses features counted: 136.

0 commit comments

Comments
 (0)