Problem or Use Case
Reasoning models whose API contract requires full reasoning_content passthrough on tool-carrying requests (DeepSeek V4 thinking mode, Kimi / Moonshot thinking mode — see agent/chat_completion_helpers.py reasoning-content padding path, refs #15250, #17400, #17341) can thrash inside a single thinking block: re-deriving the same conclusions, questioning already-derived answers, and burning large amounts of input tokens without completing the task. Reported in #78807 with four reasoning-level runs: a single 31K-char reasoning block in the Low run, a false "done" completion in the Med run, and (per the reporter) 500K+ input tokens in the High run before manual intervention. Because the guardrails only watch repeated tool calls, they never fire — the repetition happens inside one reasoning block, which the tool-call detectors don't see.
Today there is no visibility signal: the user only discovers the waste after the fact, and every subsequent turn re-sends the accumulated reasoning (contract-mandated), compounding the cost.
Proposed Solution
A per-turn reasoning-budget warning (deliberately not a kill — verbose reasoning is legitimate for some tasks):
- New config key under
agent: (e.g. reasoning_warn_after_tokens, default ~100K or disabled) — when a single turn's streamed reasoning output (reasoning_content / thinking deltas) exceeds the ceiling, Hermes emits a visible status: "model has produced N tokens of reasoning without a result — consider interrupting" and optionally appends a one-shot system-style nudge for the next turn asking the model to conclude ("You have been reasoning extensively; produce your final answer or make your planned tool call now.").
- Count reasoning output tokens from the streamed deltas; also evaluate the reporter's input-side idea: warn when reasoning output is disproportionately large relative to the prompt/context being re-sent (a ratio check), which avoids flagging long-but-appropriate reasoning on large tasks while catching thrash on small prompts.
- Applies to all reasoning providers (DeepSeek, Kimi/Moonshot, GLM, MiniMax, Qwen thinking variants, local models) — the counter is provider-agnostic since it consumes the same reasoning delta stream the loop already accumulates.
Alternatives Considered
- Reasoning-block repetition detection (hash/similarity across or within blocks) — rejected for now: high false-positive risk, since verbose reasoning models legitimately repeat structural boilerplate; no demonstrated discriminator.
- Killing the stream / hard stop on long reasoning — too aggressive; would break legitimate long thinking on complex tasks.
- Server-side reasoning summarization (send a targeted summary instead of verbatim reasoning with tool calls) — contract-blocked for DeepSeek V4 and Kimi/Moonshot: with
tools in the request, the API requires full reasoning_content passthrough and returns HTTP 400 otherwise. Could apply to providers without that requirement, but that's a larger feature.
- Existing proactive tool-result pruning (
agent.proactive_prune_tokens / proactive_prune_min_reclaim_tokens) — complementary (reclaims re-sent tool results) but does not cover reasoning blocks.
Feature Type
Configuration option
Scope
Medium (few files, < 300 lines)
Acceptance Criteria
- New config key
agent.reasoning_warn_after_tokens (or equivalent) added to config defaults, with a documented default and a disable option.
- When a single turn's streamed reasoning output exceeds the threshold, Hermes emits a visible status warning.
- Optional one-shot nudge for the next turn asking the model to conclude.
- Ratio check (reasoning output vs. prompt/context size) implemented and configurable.
- Unit tests covering both absolute-token and ratio-based triggers; docs updated.
Related Issues
Problem or Use Case
Reasoning models whose API contract requires full
reasoning_contentpassthrough on tool-carrying requests (DeepSeek V4 thinking mode, Kimi / Moonshot thinking mode — seeagent/chat_completion_helpers.pyreasoning-content padding path, refs #15250, #17400, #17341) can thrash inside a single thinking block: re-deriving the same conclusions, questioning already-derived answers, and burning large amounts of input tokens without completing the task. Reported in #78807 with four reasoning-level runs: a single 31K-char reasoning block in the Low run, a false "done" completion in the Med run, and (per the reporter) 500K+ input tokens in the High run before manual intervention. Because the guardrails only watch repeated tool calls, they never fire — the repetition happens inside one reasoning block, which the tool-call detectors don't see.Today there is no visibility signal: the user only discovers the waste after the fact, and every subsequent turn re-sends the accumulated reasoning (contract-mandated), compounding the cost.
Proposed Solution
A per-turn reasoning-budget warning (deliberately not a kill — verbose reasoning is legitimate for some tasks):
agent:(e.g.reasoning_warn_after_tokens, default ~100K or disabled) — when a single turn's streamed reasoning output (reasoning_content / thinking deltas) exceeds the ceiling, Hermes emits a visible status: "model has produced N tokens of reasoning without a result — consider interrupting" and optionally appends a one-shot system-style nudge for the next turn asking the model to conclude ("You have been reasoning extensively; produce your final answer or make your planned tool call now.").Alternatives Considered
toolsin the request, the API requires fullreasoning_contentpassthrough and returns HTTP 400 otherwise. Could apply to providers without that requirement, but that's a larger feature.agent.proactive_prune_tokens/proactive_prune_min_reclaim_tokens) — complementary (reclaims re-sent tool results) but does not cover reasoning blocks.Feature Type
Configuration option
Scope
Medium (few files, < 300 lines)
Acceptance Criteria
agent.reasoning_warn_after_tokens(or equivalent) added to config defaults, with a documented default and a disable option.Related Issues