fix(ask): raise output budget to 8192 and fail on empty strategy

CHANGELOG.md

@@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Release image gate gained a `probe` scenario (`make release-test` runs it as part of `all`): container-level checks that a Python test suite can't cover because they depend on the shipped image's process supervision — `OPEN_NOTEBOOK_WORKER_MAX_TASKS` reaching the in-image worker (the supervisord `sh -c` expansion), and the worker surviving startup with `HTTP_PROXY` set while a user's `NO_PROXY` value is preserved (the internal SurrealDB websocket not being tunneled). Both were manual probes during the v1.14.0 release; they now run automatically. Release-process docs gained the post-tag re-cut sequence and a note on never leaving the version bump uncommitted (v1.14.0 retro)
 
 ### Fixed
+- **Ask answers are no longer silently truncated.** The three Ask stages (search strategy, per-search answers, final synthesis) were capped at 2000 output tokens, well below the 8192 that chat and transformations use. Token-dense languages such as Japanese hit the cap mid-sentence, and reasoning models spent the whole budget thinking and returned blank search terms, so Ask answered "no documents found" from a corpus that had the answer. All three stages now share an 8192 budget; a strategy with no usable search terms fails with an explicit error instead of running empty searches, and thinking-only partial answers are dropped before synthesis (#1221)
 - **Remote Crawl4AI servers that require a bearer token work again.** Crawl4AI Docker ≥ 0.9.0 rejects unauthenticated external connections by default, so pointing `CRAWL4AI_API_URL` at a current instance failed URL processing. content-core is bumped to 2.0.7, which sends `CRAWL4AI_API_TOKEN` as `Authorization: Bearer`; the variable is documented in the environment reference, `.env.example` and `docker-compose.yml`. No behavior change when the token is unset (#1269, lfnovo/content-core#80)
 - **Source Chat streaming no longer drops or corrupts tokens.** The SSE reader decoded each network chunk in isolation, so a `data:` line split across two chunks was silently discarded and a multibyte character straddling a chunk boundary rendered as `�`. The reader now keeps a carry-over buffer and streaming decoder, matching the pattern Ask already used (#1289)
 - **Markdown editor follows the app theme.** The note editor's preview/edit surface was hardcoded to light mode, making it unreadable in dark mode; it now tracks the effective theme, gated on store hydration so there is no light flash before the persisted theme loads (#1294, #1268)

open_notebook/graphs/ask.py

@@ -11,11 +11,17 @@
 
 from open_notebook.ai.provision import provision_langchain_model
 from open_notebook.domain.notebook import vector_search
-from open_notebook.exceptions import OpenNotebookError
+from open_notebook.exceptions import ExternalServiceError, OpenNotebookError
 from open_notebook.utils import clean_thinking_content
 from open_notebook.utils.error_classifier import classify_error
 from open_notebook.utils.text_utils import extract_text_content
 
+# Output budget shared by the three Ask stages (strategy, per-search answers,
+# final synthesis). Matches chat and transformations. The previous 2000 cap
+# silently truncated answers in token-dense languages and left reasoning
+# models with no budget for the visible answer after their thinking (#1221).
+ASK_MAX_TOKENS = 8192
+
 
 class SubGraphState(TypedDict):
     question: str
@@ -60,7 +66,7 @@ async def call_model_with_messages(state: ThreadState, config: RunnableConfig) -
             system_prompt,
             config.get("configurable", {}).get("strategy_model"),
             "tools",
-            max_tokens=2000,
+            max_tokens=ASK_MAX_TOKENS,
             structured=dict(type="json"),
         )
         # model = model.bind_tools(tools)
@@ -74,6 +80,19 @@ async def call_model_with_messages(state: ThreadState, config: RunnableConfig) -
         # Parse the cleaned JSON content
         strategy = parser.parse(cleaned_content)
 
+        # A reasoning model that spends its whole budget thinking returns a
+        # syntactically valid strategy with blank search terms. Drop those and
+        # fail loudly when nothing usable remains, instead of running empty
+        # vector searches and answering "no documents found".
+        strategy.searches = [s for s in strategy.searches if s.term.strip()]
+        if not strategy.searches:
+            raise ExternalServiceError(
+                "The strategy model returned no search terms for this question. "
+                "This usually means the model spent its output budget on reasoning "
+                "or returned an empty response. Pick a different strategy model in "
+                "the Ask page's advanced model options, or rephrase the question."
+            )
+
         return {"strategy": strategy}
     except OpenNotebookError:
         raise
@@ -114,11 +133,15 @@ async def provide_answer(state: SubGraphState, config: RunnableConfig) -> dict:
             system_prompt,
             config.get("configurable", {}).get("answer_model"),
             "tools",
-            max_tokens=2000,
+            max_tokens=ASK_MAX_TOKENS,
         )
         ai_message = await model.ainvoke(system_prompt)
-        ai_content = extract_text_content(ai_message.content)
-        return {"answers": [clean_thinking_content(ai_content)]}
+        ai_content = clean_thinking_content(extract_text_content(ai_message.content))
+        if not ai_content.strip():
+            # Nothing left after stripping thinking content — an empty partial
+            # answer only pollutes the final synthesis.
+            return {"answers": []}
+        return {"answers": [ai_content]}
     except OpenNotebookError:
         raise
     except Exception as e:
@@ -133,7 +156,7 @@ async def write_final_answer(state: ThreadState, config: RunnableConfig) -> dict
             system_prompt,
             config.get("configurable", {}).get("final_answer_model"),
             "tools",
-            max_tokens=2000,
+            max_tokens=ASK_MAX_TOKENS,
         )
         ai_message = await model.ainvoke(system_prompt)
         final_content = extract_text_content(ai_message.content)

open_notebook/utils/text_utils.py

@@ -94,6 +94,12 @@ def parse_thinking_content(content: str) -> Tuple[str, str]:
         cleaned_content = content[malformed_match.end() :].strip()
         return thinking_content, cleaned_content
 
+    # Handle truncated output: <think>content (no closing tag). The model ran
+    # out of output budget while still reasoning, so nothing is a real answer.
+    stripped = content.lstrip()
+    if stripped.startswith("<think>"):
+        return stripped[len("<think>") :].strip(), ""
+
     return "", content
 
 

tests/test_ask_graph.py

@@ -0,0 +1,165 @@
+"""
+Unit tests for the Ask graph (open_notebook.graphs.ask).
+
+Covers the output token budget shared by the three model stages and the
+handling of empty strategies / empty partial answers produced by reasoning
+models that exhaust their budget while thinking (#1221).
+"""
+
+import json
+from typing import cast
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+from langchain_core.runnables import RunnableConfig
+
+from open_notebook.exceptions import ExternalServiceError
+from open_notebook.graphs.ask import (
+    ASK_MAX_TOKENS,
+    Search,
+    Strategy,
+    ThreadState,
+    call_model_with_messages,
+    provide_answer,
+    write_final_answer,
+)
+
+EMPTY_CONFIG = cast(RunnableConfig, {"configurable": {}})
+
+
+def _model_returning(content: str) -> MagicMock:
+    model = MagicMock()
+    model.ainvoke = AsyncMock(return_value=MagicMock(content=content))
+    return model
+
+
+def _strategy_json(terms: list[str]) -> str:
+    return json.dumps(
+        {
+            "reasoning": "look things up",
+            "searches": [{"term": t, "instructions": "extract"} for t in terms],
+        }
+    )
+
+
+class TestAskTokenBudget:
+    def test_budget_matches_other_workflows(self):
+        """Ask uses the same 8192 budget as chat and transformations."""
+        assert ASK_MAX_TOKENS == 8192
+
+    @pytest.mark.asyncio
+    async def test_strategy_stage_uses_shared_budget(self):
+        state = cast(ThreadState, {"question": "q"})
+        with patch(
+            "open_notebook.graphs.ask.provision_langchain_model",
+            new=AsyncMock(return_value=_model_returning(_strategy_json(["rag"]))),
+        ) as provision:
+            await call_model_with_messages(state, EMPTY_CONFIG)
+        assert provision.call_args.kwargs["max_tokens"] == ASK_MAX_TOKENS
+
+    @pytest.mark.asyncio
+    async def test_answer_stage_uses_shared_budget(self):
+        state = {"question": "q", "term": "rag", "instructions": "extract"}
+        with (
+            patch(
+                "open_notebook.graphs.ask.vector_search",
+                new=AsyncMock(return_value=[{"id": "source:1", "content": "x"}]),
+            ),
+            patch(
+                "open_notebook.graphs.ask.provision_langchain_model",
+                new=AsyncMock(return_value=_model_returning("partial")),
+            ) as provision,
+        ):
+            await provide_answer(state, EMPTY_CONFIG)  # type: ignore[arg-type]
+        assert provision.call_args.kwargs["max_tokens"] == ASK_MAX_TOKENS
+
+    @pytest.mark.asyncio
+    async def test_final_stage_uses_shared_budget(self):
+        state = cast(
+            ThreadState,
+            {
+                "question": "q",
+                "strategy": Strategy(reasoning="r", searches=[]),
+                "answers": ["a"],
+            },
+        )
+        with patch(
+            "open_notebook.graphs.ask.provision_langchain_model",
+            new=AsyncMock(return_value=_model_returning("final")),
+        ) as provision:
+            result = await write_final_answer(state, EMPTY_CONFIG)
+        assert provision.call_args.kwargs["max_tokens"] == ASK_MAX_TOKENS
+        assert result == {"final_answer": "final"}
+
+
+class TestEmptyStrategyHandling:
+    @pytest.mark.asyncio
+    async def test_blank_search_terms_are_dropped(self):
+        state = cast(ThreadState, {"question": "q"})
+        with patch(
+            "open_notebook.graphs.ask.provision_langchain_model",
+            new=AsyncMock(
+                return_value=_model_returning(_strategy_json(["", "  ", "rag"]))
+            ),
+        ):
+            result = await call_model_with_messages(state, EMPTY_CONFIG)
+        assert [s.term for s in result["strategy"].searches] == ["rag"]
+
+    @pytest.mark.asyncio
+    async def test_all_blank_terms_raise_instead_of_silent_no_results(self):
+        state = cast(ThreadState, {"question": "q"})
+        with patch(
+            "open_notebook.graphs.ask.provision_langchain_model",
+            new=AsyncMock(return_value=_model_returning(_strategy_json(["", "", ""]))),
+        ):
+            with pytest.raises(ExternalServiceError, match="no search terms"):
+                await call_model_with_messages(state, EMPTY_CONFIG)
+
+    @pytest.mark.asyncio
+    async def test_no_searches_raise(self):
+        state = cast(ThreadState, {"question": "q"})
+        with patch(
+            "open_notebook.graphs.ask.provision_langchain_model",
+            new=AsyncMock(return_value=_model_returning(_strategy_json([]))),
+        ):
+            with pytest.raises(ExternalServiceError):
+                await call_model_with_messages(state, EMPTY_CONFIG)
+
+    @pytest.mark.asyncio
+    async def test_thinking_only_partial_answer_is_skipped(self):
+        state = {"question": "q", "term": "rag", "instructions": "extract"}
+        with (
+            patch(
+                "open_notebook.graphs.ask.vector_search",
+                new=AsyncMock(return_value=[{"id": "source:1", "content": "x"}]),
+            ),
+            patch(
+                "open_notebook.graphs.ask.provision_langchain_model",
+                new=AsyncMock(
+                    return_value=_model_returning("<think>only reasoning</think>")
+                ),
+            ),
+        ):
+            result = await provide_answer(state, EMPTY_CONFIG)  # type: ignore[arg-type]
+        assert result == {"answers": []}
+
+    @pytest.mark.asyncio
+    async def test_truncated_thinking_partial_answer_is_skipped(self):
+        """Budget exhausted inside <think> must not leak reasoning as an answer."""
+        state = {"question": "q", "term": "rag", "instructions": "extract"}
+        with (
+            patch(
+                "open_notebook.graphs.ask.vector_search",
+                new=AsyncMock(return_value=[{"id": "source:1", "content": "x"}]),
+            ),
+            patch(
+                "open_notebook.graphs.ask.provision_langchain_model",
+                new=AsyncMock(return_value=_model_returning("<think>cut off mid")),
+            ),
+        ):
+            result = await provide_answer(state, EMPTY_CONFIG)  # type: ignore[arg-type]
+        assert result == {"answers": []}
+
+    def test_search_model_accepts_blank_term(self):
+        """The filter, not the schema, is responsible for blank terms."""
+        assert Search(term="", instructions="x").term == ""

tests/test_utils.py

@@ -108,6 +108,14 @@ def test_parse_thinking_content_malformed_no_open_tag(self):
         assert thinking == "Some thinking content"
         assert cleaned == "Here is my answer"
 
+    def test_parse_thinking_content_truncated_no_close_tag(self):
+        """Output cut off inside <think> is all reasoning, never an answer."""
+        content = "<think>Let me reason about this at length and then"
+        thinking, cleaned = parse_thinking_content(content)
+        assert thinking == "Let me reason about this at length and then"
+        assert cleaned == ""
+        assert clean_thinking_content("  <think>partial") == ""
+
     def test_parse_thinking_content_invalid_input(self):
         """Test parsing with invalid input types."""
         # Non-string input (intentionally violates the signature to test the
@@ -472,14 +480,10 @@ def test_notice_only_budget_omits_source(self):
             "full_text": "e" + SOURCE_TRUNCATION_NOTICE,
         }
         notice_tokens = token_count(
-            _format_source_context(
-                {"sources": [notice_only], "insights": []}
-            )
+            _format_source_context({"sources": [notice_only], "insights": []})
         )
         one_character_tokens = token_count(
-            _format_source_context(
-                {"sources": [one_character], "insights": []}
-            )
+            _format_source_context({"sources": [one_character], "insights": []})
         )
         assert notice_tokens < one_character_tokens