fix(middleware): handle orphaned tool results in DanglingToolCallMiddleware#1973
Open
octo-patch wants to merge 2 commits intobytedance:mainfrom
Open
fix(middleware): handle orphaned tool results in DanglingToolCallMiddleware#1973octo-patch wants to merge 2 commits intobytedance:mainfrom
octo-patch wants to merge 2 commits intobytedance:mainfrom
Conversation
…nt context Reasoning models (e.g. minimax M2.7, DeepSeek-R1) emit <think>...</think> blocks before their actual output. When such a model is used as the title model (or as the main agent), the raw thinking content leaked into the thread title stored in state, so the chat list showed the internal monologue instead of a meaningful title. Fixes bytedance#1884 - Add `_strip_think_tags()` helper using a regex to remove all <think>...</think> blocks - Apply it in `_parse_title()` so the title model response is always clean - Apply it to the assistant message in `_build_title_prompt()` so thinking content from the first AI turn is not fed back to the title model - Add four new unit tests covering: stripping in parse, think-only response, assistant prompt stripping, and end-to-end async flow with think tags
…leware When a tool call cycle completes (AIMessage → ToolMessage(s)) but the user sends a follow-up HumanMessage before the AI has a chance to respond to the tool results, the conversation history becomes malformed: AI(tool_calls=[A]) → Tool(A) → HumanMessage Many LLMs require an AIMessage between ToolMessages and HumanMessages, so this pattern causes persistent LLM errors that cannot be recovered from. Fix: add a Phase 2 pass to _build_patched_messages that detects any ToolMessage directly followed by a HumanMessage and injects a placeholder AIMessage between them, similar to how Phase 1 handles dangling tool calls. Closes bytedance#1936
Collaborator
|
@octo-patch this PR has two unrelated commits. Please keep commits clean so we can apply them at any time. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1936
Problem
DanglingToolCallMiddlewarealready handles the case where anAIMessagehastool_callsbut no correspondingToolMessageexists (dangling tool call). However, it did not handle a related but distinct problem:When a tool call cycle completes (AIMessage → ToolMessage(s)) but the user sends a follow-up
HumanMessagebefore the AI has a chance to respond to the tool results, the conversation history becomes:Many LLMs require an
AIMessagebetweenToolMessagesandHumanMessages. Without it, every subsequent model invocation fails with errors that cannot be recovered from, causing the session to be permanently broken (as reported in the issue).Solution
Add a Phase 2 pass to
_build_patched_messagesthat scans the (Phase 1 patched) message list and injects a placeholderAIMessageimmediately after anyToolMessagethat is directly followed by aHumanMessage.The two-phase approach ensures correct ordering:
ToolMessages for dangling tool calls (existing behavior)AIMessages for orphaned tool results (new behavior)Phase 2 does not inject when a
ToolMessageis at the end of the list (the model is about to generate a real response) or when it is already followed by anAIMessage(valid conversation flow).Testing
AIMessageafter the Phase 1 syntheticToolMessage)TestOrphanedToolResults:AIMessageinjected)AIMessage(no Phase 2 patch needed)