Skip to content

fix(ask): raise output budget to 8192 and fail on empty strategy - #1323

Merged
lfnovo merged 3 commits into
mainfrom
fix/issue-1221-ask-max-tokens
Sep 5, 2026
Merged

fix(ask): raise output budget to 8192 and fail on empty strategy#1323
lfnovo merged 3 commits into
mainfrom
fix/issue-1221-ask-max-tokens

Conversation

@lfnovo

@lfnovo lfnovo commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

The three Ask stages (search strategy, per-search answers, final synthesis) in open_notebook/graphs/ask.py were hardcoded to max_tokens=2000, well below the 8192 that chat and transformations already use. Two failure modes followed from that, both returning HTTP 200:

  • Token-dense languages (Japanese, etc.) hit the cap and the answer was cut mid-sentence.
  • Reasoning models (Qwen3 on Ollama, thinking-enabled Claude/GPT) spent the whole budget thinking. After clean_thinking_content() the strategy had blank search terms, vector search returned nothing, and Ask answered "no documents found" from a corpus that contained the answer.

This PR:

Closes #1221

Test plan

  • uv run pytest tests/ — 679 passed (9 new tests in tests/test_ask_graph.py cover the budget on all three stages, blank-term filtering, the empty-strategy error and the empty partial-answer skip).
  • uv run ruff check . — clean.
  • uv run python -m mypy . — clean.

Review in cubic

The three Ask stages were capped at max_tokens=2000, far below the 8192
used by chat and transformations. Token-dense languages hit the cap
mid-sentence and reasoning models spent the whole budget thinking,
returning blank search terms and a misleading "no documents found".

- Share an 8192 budget across strategy, per-search answers and final
  synthesis (ASK_MAX_TOKENS).
- Drop blank search terms; raise ExternalServiceError when none remain.
- Skip thinking-only partial answers before final synthesis.

Closes #1221

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GH6spwym6qmmdmxCe5CXfu

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread open_notebook/graphs/ask.py
Comment thread open_notebook/graphs/ask.py Outdated
… model options

Addresses cubic review on #1323:
- parse_thinking_content now treats content that opens <think> without
  closing it (budget exhausted mid-reasoning) as thinking only, so no
  reasoning leaks into partial answers.
- Empty-strategy error message points at the Ask page's advanced model
  options, where the strategy model is actually chosen.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GH6spwym6qmmdmxCe5CXfu
@lfnovo

lfnovo commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

Addressed both findings: parse_thinking_content now treats an unclosed <think> block as reasoning-only (so a truncated partial answer is dropped rather than leaked), and the empty-strategy error now points at the Ask page's advanced model options. Tests added for both.

@cubic-dev-ai review please

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Addressed both findings: parse_thinking_content now treats an unclosed <think> block as reasoning-only (so a truncated partial answer is dropped rather than leaked), and the empty-strategy error now points at the Ask page's advanced model options. Tests added for both.

@cubic-dev-ai review please

@lfnovo I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Ask (Q&A) answers are silently truncated due to hardcoded max_tokens=2000 in graphs/ask.py — especially visible in non-English languages

1 participant