@@ -19,13 +19,13 @@ inference provider, based on integration test results.
1919
2020| Provider | Tested | Passing | Failing | Coverage |
2121| ----------| --------| ---------| ---------| ----------|
22- | azure | 111 | 111 | 0 | 81 % |
22+ | azure | 111 | 111 | 0 | 82 % |
2323| bedrock | 27 | 27 | 0 | 20% |
24- | ollama | 4 | 4 | 0 | 3 % |
25- | openai | 136 | 136 | 0 | 99 % |
26- | vertexai | 70 | 70 | 0 | 51 % |
27- | vllm | 4 | 4 | 0 | 3 % |
28- | watsonx | 61 | 61 | 0 | 44 % |
24+ | ollama | 3 | 3 | 0 | 2 % |
25+ | openai | 136 | 136 | 0 | 100 % |
26+ | vertexai | 70 | 70 | 0 | 52 % |
27+ | vllm | 3 | 3 | 0 | 2 % |
28+ | watsonx | 61 | 61 | 0 | 45 % |
2929
3030## Provider Details
3131
@@ -35,7 +35,7 @@ Models, endpoints, and versions used during test recordings.
3535| ----------| ----------| ----------| --------------|
3636| azure | gpt-4o | llama-stack-test.openai.azure.com, lls-test.openai.azure.com, ogx-test.openai.azure.com | openai sdk: 2.43.0 |
3737| bedrock | openai.gpt-oss-20b-1:0 | bedrock-runtime.us-west-2.amazonaws.com | openai sdk: 2.30.0 |
38- | ollama | deepseek-r1:1.5b, gpt-oss:20b | — | openai sdk: 2.43.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
38+ | ollama | deepseek-r1:1.5b | — | openai sdk: 2.43.0 |
3939| openai | gpt-4o, o4-mini, text-embedding-3-small | api.openai.com | openai sdk: 2.43.0 |
4040| vertexai | publishers/google/models/gemini-2.0-flash | — | openai sdk: 2.5.0, provider: vertexai |
4141| vllm | Qwen/Qwen3-0.6B | — | openai sdk: 2.5.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
@@ -167,7 +167,6 @@ Models, endpoints, and versions used during test recordings.
167167| --- | --- | --- | --- | --- | --- | --- | --- |
168168| reasoning basic streaming | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
169169| reasoning multi turn passthrough | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |
170- | reasoning multi turn with tool call | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ | ⏭️ |
171170| reasoning no summary without request | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
172171| reasoning non streaming | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |
173172| reasoning summary event ordering | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
0 commit comments