refactor: clean up reasoning implementation (#5429)

## Summary

Follow-up cleanups to #5206 (reasoning output in responses API) and
#5407 (type fixes), addressing items from [mattf's
review](#5206 (comment)).

**Code changes**

- **Extracted shared reasoning util**
(`src/llama_stack/providers/utils/inference/reasoning.py`): All three
providers (vLLM, Ollama, Bedrock) had identical message mapping and
chunk wrapping code. Pulled it into a shared `reasoning.py` util with no
defaults -- each provider explicitly passes its field names.

- **Simplified return type**: `openai_chat_completions_with_reasoning`
is streaming-only across all providers. Removed
`OpenAIChatCompletionWithReasoning` from return type in the API
protocol, router, and all provider implementations.

- **Removed dead code**: Deleted no-op `_prepare_reasoning_params` stubs
from vLLM and Bedrock providers. Inlined Ollama's version (2-line
default for `reasoning_effort`).

- **Removed type ignores**: Eliminated `# type: ignore` on
`_separate_tool_calls` assignment (reordered if/else to assign base type
first) and on `tool_call.index` usage in streaming chunk processing.

**Test changes:**
- Added `test_reasoning_basic_streaming` and
`test_reasoning_multi_turn_with_tool_call` to the `ollama-reasoning`
test suite.
- Un-skipped vLLM reasoning tests -- the CI action.yaml now configures a
reasoning parser, so vLLM actually returns reasoning tokens.



## Addressed from
[review](#5206 (comment))

| Item | Status |
|------|--------|
| `openai_chat_completions_with_reasoning` is streaming only, remove
non-streaming return type | Done |
| Remove unused `_prepare_reasoning_params` from vLLM | Done |
| Remove unused `_prepare_reasoning_params` from Bedrock | Done |
| Inline `_prepare_reasoning_params` for Ollama | Done |
| bedrock, vllm, ollama need same transformations -- consider a util |
Done |
| Put `message = OpenAIAssistantMessageParam` case first to resolve type
ignore | Done |
| Figure out why there are new type ignores | Done (removed 5 type
ignores) |
| Type of `completion_result` confused by storing both CC and
CC-with-reasoning | Done (removed non-streaming type from union) |

## TODO  ( needs design discussion )

| Item | Question |
|------|----------|
| **Non-adjacent reasoning items** | `_get_preceding_reasoning` only
checks `index - 1`, based on what I observed from official openai's
response output. Is there a missing Edge case ? |
| **Error vs silent fallback** | When a user asks for reasoning and the
provider doesn't support it, should we return an error or silently
proceed without reasoning? Currently falls back silently with a
`critical` log. |
| **Move merge logic to provider** | Suggestions Appreciated ... |
| **Gemini implementation** | Gemini is strict on input shape and may
use different field names (e.g. `"thinking"`). The shared util now
supports custom field names, so this is ready to implement. |
| **`n != 1` simplification** | Not sure what this means   |
| **Reduce `model_copy()` calls** | For safety, currently copying params
in multiple places. Need to trace mutation paths to determine redundant
copies. |
| **Complete types on `_separate_tool_calls`** | Didn't tamper for now |

## Test plan
- [x] BFCL Eval maintains accuracy
- [x] vllm server renders prompt correctly with reasoning inside
`analysis` channel
- [x] `uv run pre-commit run mypy-full --hook-stage manual --all-files`
# passes

---------

Signed-off-by: robinnarsinghranabhat <robinnarsingha123@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Author: 3 people

docs/docs/api-openai/provider_matrix.md

@@ -21,7 +21,7 @@ inference provider, based on integration test results.
 |----------|--------|---------|---------|----------|
 | azure | 111 | 111 | 0 | 82% |
 | bedrock | 27 | 27 | 0 | 20% |
-| ollama | 2 | 2 | 0 | 2% |
+| ollama | 3 | 3 | 0 | 2% |
 | openai | 136 | 136 | 0 | 100% |
 | vertexai | 70 | 70 | 0 | 52% |
 | vllm | 3 | 3 | 0 | 2% |
@@ -35,10 +35,10 @@ Models, endpoints, and versions used during test recordings.
 |----------|----------|----------|--------------|
 | azure | gpt-4o | llama-stack-test.openai.azure.com, lls-test.openai.azure.com, ogx-test.openai.azure.com | openai sdk: 2.43.0 |
 | bedrock | openai.gpt-oss-20b-1:0 | bedrock-runtime.us-west-2.amazonaws.com | openai sdk: 2.30.0 |
-| ollama | deepseek-r1:1.5b | — | openai sdk: 2.30.0 |
+| ollama | deepseek-r1:1.5b | — | openai sdk: 2.43.0 |
 | openai | gpt-4o, o4-mini, text-embedding-3-small | api.openai.com | openai sdk: 2.43.0 |
 | vertexai | publishers/google/models/gemini-2.0-flash | — | openai sdk: 2.5.0, provider: vertexai |
-| vllm | Qwen/Qwen3-0.6B | — | openai sdk: 2.5.0, vllm server: 0.18.1rc1.dev197+g0e9358c11 |
+| vllm | Qwen/Qwen3-0.6B | — | openai sdk: 2.5.0, vllm server: 0.17.2rc1.dev79+g5ce2d10e4 |
 | watsonx | meta-llama/llama-3-3-70b-instruct | us-south.ml.cloud.ibm.com | openai sdk: 2.5.0 |
 
 ## Basic Responses
@@ -165,7 +165,7 @@ Models, endpoints, and versions used during test recordings.
 
 | Feature | azure | bedrock | ollama | openai | vertexai | vllm | watsonx |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| reasoning basic streaming | ✅ | ✅ | ⏭️ | ✅ | ✅ | ✅ | ✅ |
+| reasoning basic streaming | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | reasoning multi turn passthrough | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |
 | reasoning no summary without request | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
 | reasoning non streaming | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |

src/ogx/core/routers/inference.py

@@ -45,7 +45,6 @@
     OpenAIChatCompletionResponseMessage,
     OpenAIChatCompletionToolCall,
     OpenAIChatCompletionToolCallFunction,
-    OpenAIChatCompletionWithReasoning,
     OpenAIChoice,
     OpenAIChoiceLogprobs,
     OpenAICompletion,
@@ -316,7 +315,7 @@ async def openai_chat_completion(
     async def openai_chat_completions_with_reasoning(
         self,
         params: OpenAIChatCompletionRequestWithExtraBody,
-    ) -> OpenAIChatCompletionWithReasoning | AsyncIterator[OpenAIChatCompletionChunkWithReasoning]:
+    ) -> AsyncIterator[OpenAIChatCompletionChunkWithReasoning]:
         """Called by the Responses layer when a user requests reasoning.
 
         Routes to the provider's reasoning-aware CC implementation, which

src/ogx/providers/inline/responses/builtin/responses/streaming.py

@@ -569,11 +569,12 @@ async def create_response(self) -> AsyncIterator[OpenAIResponseObjectStream]:
                 )
                 # Use reasoning-aware method when reasoning is explicitly requested
                 completion_result: (
-                    OpenAIChatCompletion
-                    | AsyncIterator[OpenAIChatCompletionChunk]
-                    | OpenAIChatCompletionWithReasoning
+                    OpenAIChatCompletionWithReasoning
+                    | OpenAIChatCompletion
                     | AsyncIterator[OpenAIChatCompletionChunkWithReasoning]
-                )
+                    | AsyncIterator[OpenAIChatCompletionChunk]
+                    | None
+                ) = None
                 if self.reasoning and self.reasoning.effort and self.reasoning.effort != "none":
                     try:
                         # Pass a copy — the router mutates params.model (strips provider prefix).
@@ -598,6 +599,7 @@ async def create_response(self) -> AsyncIterator[OpenAIResponseObjectStream]:
                         completion_result_data = stream_event_or_result
                     else:
                         yield stream_event_or_result
+
                 # If violation detected, skip the rest of processing since we already sent refusal
                 if self.violation_detected:
                     return
@@ -810,20 +812,18 @@ def _separate_tool_calls(
 
         for choice in current_response.choices:
             # Convert response message to input message format for multi-turn.
-            # Use AssistantMessageWithReasoning if reasoning was present in the
-            # CC response. Providers will be check for this AssistantMessageWithReasoning
-            # message
+            # Assign base type first, then narrow to AssistantMessageWithReasoning
+            # if reasoning was present in the CC response.
+            message: OpenAIAssistantMessageParam | AssistantMessageWithReasoning = OpenAIAssistantMessageParam(
+                content=choice.message.content,
+                tool_calls=choice.message.tool_calls,
+            )
             if reasoning_content:
                 message = AssistantMessageWithReasoning(
                     content=choice.message.content,
                     tool_calls=choice.message.tool_calls,
                     reasoning_content=reasoning_content,
                 )
-            else:
-                message = OpenAIAssistantMessageParam(  # type: ignore[assignment]
-                    content=choice.message.content,
-                    tool_calls=choice.message.tool_calls,
-                )
             next_turn_messages.append(message)
             logger.debug("Choice message content", content=choice.message.content)
             logger.debug("Choice message tool_calls", tool_calls=choice.message.tool_calls)
@@ -1239,7 +1239,7 @@ async def _process_streaming_chunks(
                 # chunk: OpenAIChatCompletionChunk annotation above.
                 if chunk_choice.delta.tool_calls:
                     for tool_call in chunk_choice.delta.tool_calls:
-                        response_tool_call = chat_response_tool_calls.get(tool_call.index, None)  # type: ignore[arg-type]
+                        response_tool_call = chat_response_tool_calls.get(tool_call.index, None)
                         # Create new tool call entry if this is the first chunk for this index
                         is_new_tool_call = response_tool_call is None
                         if is_new_tool_call:
@@ -1251,16 +1251,16 @@ async def _process_streaming_chunks(
                             if tool_call_dict.get("function") and tool_call_dict["function"].get("arguments") is None:
                                 tool_call_dict["function"]["arguments"] = ""
                             response_tool_call = OpenAIChatCompletionToolCall(**tool_call_dict)
-                            chat_response_tool_calls[tool_call.index] = response_tool_call  # type: ignore[index]
+                            chat_response_tool_calls[tool_call.index] = response_tool_call
 
                             # Create item ID for this tool call for streaming events
                             tool_call_item_id = f"fc_{uuid.uuid4()}"
-                            tool_call_item_ids[tool_call.index] = tool_call_item_id  # type: ignore[index]
+                            tool_call_item_ids[tool_call.index] = tool_call_item_id
 
                             # Emit output_item.added event for the new function call
                             self.sequence_number += 1
-                            is_mcp_tool = tool_call.function.name and tool_call.function.name in self.mcp_tool_to_server  # type: ignore[union-attr]
-                            if not is_mcp_tool and tool_call.function.name not in _SERVER_SIDE_BUILTIN_TOOL_NAMES:  # type: ignore[union-attr]
+                            is_mcp_tool = tool_call.function.name and tool_call.function.name in self.mcp_tool_to_server
+                            if not is_mcp_tool and tool_call.function.name not in _SERVER_SIDE_BUILTIN_TOOL_NAMES:
                                 # for MCP tools (and even other non-function tools) we emit an output message item later
                                 function_call_item = OpenAIResponseOutputMessageFunctionToolCall(
                                     arguments="",  # Will be filled incrementally via delta events

tests/integration/TARGET_MODELS.md

@@ -22,7 +22,7 @@ These jobs come from the `default` section of `ci_matrix.json`. They all run in
 | `bedrock-responses` | `bedrock` | 6 roots; Responses coverage: 27/136 (20%) |
 | `base-vllm-subset` | `vllm` | `inference` only |
 | `vllm-reasoning` | `vllm` | `test_reasoning.py` only; Responses coverage: 3/136 (2%) |
-| `ollama-reasoning` | `ollama-reasoning` | 3 roots; Responses coverage: 2/136 (1%) |
+| `ollama-reasoning` | `ollama-reasoning` | 5 roots; Responses coverage: 3/136 (2%) |
 | `messages` | `ollama` | `messages` only |
 | `messages-openai` | `gpt` | `messages` only |
 | `interactions` | `gemini` | `interactions` only |
@@ -80,7 +80,7 @@ This section is derived from the same replay recordings used to generate `docs/d
 | Vertex AI | 70 | 70 | 51% |
 | WatsonX | 61 | 61 | 45% |
 | Bedrock | 27 | 27 | 20% |
+| Ollama | 3 | 3 | 2% |
 | vLLM | 3 | 3 | 2% |
-| Ollama | 2 | 2 | 1% |
 
 Total Responses features counted: 136.