@@ -21,9 +21,9 @@ inference provider, based on integration test results.
2121
2222| Provider | Tested | Passing | Failing | Coverage |
2323| ----------| --------| ---------| ---------| ----------|
24- | azure | 101 | 101 | 0 | 86% |
24+ | azure | 102 | 102 | 0 | 86% |
2525| bedrock | 25 | 25 | 0 | 21% |
26- | openai | 118 | 118 | 0 | 100% |
26+ | openai | 119 | 119 | 0 | 100% |
2727| vllm | 1 | 1 | 0 | 1% |
2828| watsonx | 56 | 56 | 0 | 47% |
2929
@@ -86,58 +86,11 @@ Models, endpoints, and versions used during test recordings.
8686
8787## Openai Responses
8888
89- <<<<<<< HEAD
90- | Feature | azure | openai | vllm | watsonx |
91- | --- | --- | --- | --- | --- |
92- | background false is synchronous | ✅ | ✅ | — | ✅ |
93- | background returns queued | ✅ | ✅ | — | ✅ |
94- | cancel completed response fails | ✅ | ✅ | — | ⏭️ |
95- | incomplete details length | ✅ | ✅ | — | ✅ |
96- | incomplete details length streaming | ✅ | ✅ | — | ✅ |
97- | incomplete details max iterations exceeded | ✅ | ✅ | — | ✅ |
98- | incomplete details max iterations exceeded streaming | ✅ | ✅ | — | ✅ |
99- | incomplete details null when completed | ✅ | ✅ | — | ✅ |
100- | reasoning effort | ⏭️ | ✅ | — | ⏭️ |
101- | reasoning effort streaming | ⏭️ | ✅ | — | ⏭️ |
102- | streaming includes usage | ✅ | ✅ | — | ✅ |
103- | streaming invalid base64 image failure code is spec compliant | ✅ | ✅ | — | ⏭️ |
104- | with max output tokens | ✅ | ✅ | — | ⏭️ |
105- | with parallel tool calls and previous response | ✅ | ✅ | — | ✅ |
106- | with parallel tool calls disabled | ✅ | ✅ | — | ⏭️ |
107- | with parallel tool calls disabled streaming | ✅ | ✅ | — | ⏭️ |
108- | with parallel tool calls enabled | ✅ | ✅ | — | ⏭️ |
109- | with prompt cache key | ✅ | ✅ | — | ✅ |
110- | with prompt cache key and previous response | ✅ | ✅ | — | ✅ |
111- | with prompt cache key streaming | ✅ | ✅ | — | ✅ |
112- | with safety identifier | ✅ | ✅ | — | ✅ |
113- | with safety identifier and previous response | ✅ | ✅ | — | ✅ |
114- | with safety identifier streaming | ✅ | ✅ | — | ✅ |
115- | with service tier | ⏭️ | ✅ | — | ⏭️ |
116- | with service tier and previous response | ⏭️ | ✅ | — | ⏭️ |
117- | with service tier auto | ⏭️ | ✅ | — | ⏭️ |
118- | with service tier auto and previous response | ⏭️ | ✅ | — | ⏭️ |
119- | with service tier auto streaming | ⏭️ | ✅ | — | ⏭️ |
120- | with service tier flex | ⏭️ | ✅ | — | ⏭️ |
121- | with service tier flex streaming | ⏭️ | ✅ | — | ⏭️ |
122- | with service tier streaming | ⏭️ | ✅ | — | ⏭️ |
123- | with small max output tokens | ✅ | ✅ | — | ⏭️ |
124- | with stream options and previous response | ✅ | ✅ | — | ✅ |
125- | with stream options includes usage | ✅ | ✅ | — | ✅ |
126- | with stream options non streaming | ✅ | ✅ | — | ✅ |
127- | with top logprobs | ✅ | ✅ | — | ✅ |
128- | with top logprobs and previous response | ✅ | ✅ | — | ✅ |
129- | with top logprobs streaming | ✅ | ✅ | — | ✅ |
130- | with top p | ✅ | ✅ | — | ✅ |
131- | with top p and previous response | ✅ | ✅ | — | ✅ |
132- | with top p streaming | ✅ | ✅ | — | ✅ |
133- | with truncation and previous response | ✅ | ✅ | — | ✅ |
134- | with truncation disabled | ✅ | ✅ | — | ✅ |
135- | with truncation disabled streaming | ✅ | ✅ | — | ✅ |
136- =======
13789| Feature | azure | bedrock | openai | vllm | watsonx |
13890| --- | --- | --- | --- | --- | --- |
13991| background false is synchronous | ✅ | — | ✅ | — | ✅ |
14092| background returns queued | ✅ | — | ✅ | — | ✅ |
93+ | cancel completed response fails | ✅ | — | ✅ | — | ⏭️ |
14194| incomplete details length | ✅ | — | ✅ | — | ✅ |
14295| incomplete details length streaming | ✅ | — | ✅ | — | ✅ |
14396| incomplete details max iterations exceeded | ✅ | — | ✅ | — | ✅ |
@@ -179,7 +132,6 @@ Models, endpoints, and versions used during test recordings.
179132| with truncation and previous response | ✅ | — | ✅ | — | ✅ |
180133| with truncation disabled | ✅ | — | ✅ | — | ✅ |
181134| with truncation disabled streaming | ✅ | — | ✅ | — | ✅ |
182- >>>>>>> 2cc1f670b (feat: add Bedrock to responses CI suite with recordings (#5254 ))
183135
184136## Prompt Templates
185137
0 commit comments