fix(ask): raise output budget to 8192 and fail on empty strategy - #1323
Conversation
The three Ask stages were capped at max_tokens=2000, far below the 8192 used by chat and transformations. Token-dense languages hit the cap mid-sentence and reasoning models spent the whole budget thinking, returning blank search terms and a misleading "no documents found". - Share an 8192 budget across strategy, per-search answers and final synthesis (ASK_MAX_TOKENS). - Drop blank search terms; raise ExternalServiceError when none remain. - Skip thinking-only partial answers before final synthesis. Closes #1221 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GH6spwym6qmmdmxCe5CXfu
There was a problem hiding this comment.
All reported issues were addressed across 3 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
… model options Addresses cubic review on #1323: - parse_thinking_content now treats content that opens <think> without closing it (budget exhausted mid-reasoning) as thinking only, so no reasoning leaks into partial answers. - Empty-strategy error message points at the Ask page's advanced model options, where the strategy model is actually chosen. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GH6spwym6qmmdmxCe5CXfu
|
Addressed both findings: @cubic-dev-ai review please |
@lfnovo I have started the AI code review. It will take a few minutes to complete. |
Summary
The three Ask stages (search strategy, per-search answers, final synthesis) in
open_notebook/graphs/ask.pywere hardcoded tomax_tokens=2000, well below the 8192 that chat and transformations already use. Two failure modes followed from that, both returning HTTP 200:clean_thinking_content()the strategy had blank search terms, vector search returned nothing, and Ask answered "no documents found" from a corpus that contained the answer.This PR:
ASK_MAX_TOKENS = 8192budget across the three stages, aligned withgraphs/chat.pyandgraphs/transformation.py. No new environment variable: per-model tunables are being consolidated under [Design] Model & Provider Configuration (advanced params, presets, health) #947 / feat(chat): expose temperature/top_p controls #830, so this keeps the fix to a sane default in the meantime (same reasoning as the parking note on fix: make Ask output token budget configurable #1247).ExternalServiceErrorwith an actionable message when none remain, instead of running empty searches.Closes #1221
Test plan
uv run pytest tests/— 679 passed (9 new tests intests/test_ask_graph.pycover the budget on all three stages, blank-term filtering, the empty-strategy error and the empty partial-answer skip).uv run ruff check .— clean.uv run python -m mypy .— clean.