Skip to content

Commit 3b7243d

Browse files
authored
fix(sources): fall back to auto when a selected engine's runtime is absent (#1194)
* fix(sources): fall back to auto when a selected engine's runtime is absent The content-processing engine choice is persisted in the database; the runtime that serves it (Docling, local Crawl4AI) is installed on demand from environment flags evaluated at boot. The two therefore drift: a redeploy that drops OPEN_NOTEBOOK_ENABLE_CRAWL4AI/_DOCLING, a volume moved to a new deployment, or a failed on-demand install all leave a stored selection pointing at a runtime that is not there. The source graph passed that selection straight to content-core, so every affected extraction failed with "Could not extract any text content from this source" - no mention of the engine, the runtime, or the flag that would fix it. For a URL engine set to crawl4ai this breaks URL ingestion entirely. The graph now checks runtime availability before honoring the stored engine and degrades to content-core's "auto" chain, logging a WARNING that names the engine and the env var that would enable it. Engines with no opt-in runtime (auto/simple/firecrawl/jina) are passed through untouched. The availability probes moved from api/routers/capabilities.py to open_notebook/utils/runtime_capabilities.py so the graph can use them without importing from the API layer; the capabilities endpoint keeps identical behavior and its tests follow the probes to their new home. Found by the smoke-e2e agent during v1.14.0 release testing, on a dev environment that was in exactly this state. Pre-existing since v1.13.0 (#1122 made the runtimes opt-in, #432 made the stored selection take effect), not a v1.14.0 regression. * docs(changelog): record the unavailable-engine fallback fix
1 parent 7cac3da commit 3b7243d

9 files changed

Lines changed: 357 additions & 100 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [Unreleased]
8+
## [1.14.0] - 2026-07-20
99

1010
### Added
1111
- **Docling formula & vision enrichment toggles** in Settings → Content Processing. Two new opt-in checkboxes surface content-core 2.x's `docling_formulas` (extract mathematical formulas as structured markup) and `docling_vision` (describe images and extract chart data with a vision model) enrichment flags, mirroring the existing OCR toggle. Both default off; the vision help text warns it is significantly slower and may call a vision model. Like OCR, the toggles only take effect through the Docling engine, so they are gated on Docling availability. The settings persist via `GET`/`PUT /api/settings` and are threaded into content-core extraction alongside `docling_ocr` (migration 23 backfills the new fields); labels and help are translated across all 14 locales (#1131)
@@ -28,6 +28,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2828
- Japanese (ja-JP) translations reviewed and improved throughout: "トランスフォーメーション" replaced with the natural 変換, duration formatting given proper spacing, and accessibility labels, error messages and assorted phrasings rewritten to read naturally rather than as literal translations (#998)
2929

3030
### Fixed
31+
- **A content-processing engine whose runtime is not installed no longer breaks ingestion silently.** The engine choice is persisted in the database, but the opt-in runtimes that serve it (Docling, local Crawl4AI) are installed from environment flags evaluated at boot — so a redeploy that drops `OPEN_NOTEBOOK_ENABLE_CRAWL4AI`/`_DOCLING`, a data volume moved to a new deployment, or a failed on-demand install leaves a stored selection pointing at a runtime that is absent. That selection was passed straight to content-core, and every affected extraction failed with "Could not extract any text content from this source" — naming neither the engine, the runtime, nor the flag that would fix it; with the URL engine set to Crawl4AI it broke URL ingestion entirely. The source graph now checks runtime availability first and degrades to content-core's `auto` chain, logging a warning that names the engine and the variable that would enable it; engines needing no opt-in runtime (`auto`/`simple`/`firecrawl`/`jina`) are untouched. The availability probes moved to `open_notebook/utils/runtime_capabilities.py` so the graph can share them with `GET /api/capabilities`, whose behavior is unchanged (#1194)
3132
- **HTTP proxy no longer breaks worker/API startup.** `websockets` 15.0 began auto-detecting `HTTP_PROXY`/`HTTPS_PROXY` and tunnels even `ws://` connections through the proxy, so with a proxy set the internal SurrealDB websocket (`ws://host.docker.internal:8018/rpc` or `ws://surrealdb:8000/rpc`) was routed through the external proxy, which rejected the internal host with HTTP 403 and killed the worker on startup. Open Notebook now injects `host.docker.internal,surrealdb,localhost,127.0.0.1` into `no_proxy`/`NO_PROXY` at startup (merged with any user value, never clobbering it), and the `.env.example` / docs `NO_PROXY` examples now include the internal DB hosts (#1160)
3233
- Auto-assign no longer silently re-populates optional model defaults that were deliberately cleared. Previously `POST /models/auto-assign` treated every empty slot as "missing" and filled it, so an optional slot a user intentionally cleared (to fall back to the chat model) got re-assigned on the next run. Auto-assign now fills only the two required slots (chat, embedding); the optional slots (transformation, tools, large context, TTS, STT) are left untouched. `large_context` now also falls back to the chat model when unset (matching transformation/tools) instead of returning nothing, and the Settings UI shows an inline hint on each empty optional slot — "using chat model (…)" for the text slots, "not configured" for TTS/STT (#1098)
3334
- PPQ model discovery now lists all modalities, not just chat. PPQ's `/v1/models` returns only chat/language models by default; the discovery URL now requests `?type=all` so the embedding, speech-to-text and text-to-speech models this multi-modality gateway advertises actually surface in the provider matrix (`PPQ_MODEL_TYPES` classifies them into the right slots) (#1180)

api/routers/capabilities.py

Lines changed: 12 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -8,101 +8,33 @@
88
docs/7-DEVELOPMENT/decisions/ADR-007-optin-runtimes.md), so this endpoint probes
99
what is *actually* importable/reachable rather than trusting the enable flags.
1010
11+
The probes themselves live in open_notebook.utils.runtime_capabilities because
12+
the source-processing graph needs the same signal (it must not pass content-core
13+
an engine whose runtime is absent).
14+
1115
Endpoints:
1216
- GET /capabilities - Availability of Docling and Crawl4AI runtimes
1317
"""
1418

15-
import importlib.util
16-
import os
17-
import sys
18-
1919
from fastapi import APIRouter
20-
from loguru import logger
2120

2221
from api.models import CapabilitiesResponse
22+
from open_notebook.utils.runtime_capabilities import (
23+
crawl4ai_local_ready,
24+
crawl4ai_remote_configured,
25+
docling_available,
26+
)
2327

2428
router = APIRouter(prefix="/capabilities", tags=["capabilities"])
2529

2630

27-
def _docling_available() -> bool:
28-
"""True when Docling is installed (its document engine, OCR and image sources work)."""
29-
try:
30-
# content-core's own routing gate — the authoritative signal, not just a spec check.
31-
from content_core.extraction import DOCLING_AVAILABLE
32-
33-
return bool(DOCLING_AVAILABLE)
34-
except (ImportError, AttributeError):
35-
# Content-core absent or its API moved — fall back to a plain spec check.
36-
return importlib.util.find_spec("docling") is not None
37-
except Exception:
38-
# An unexpected failure shouldn't be silently masked as "unavailable".
39-
logger.opt(exception=True).warning(
40-
"Unexpected error probing Docling availability; reporting unavailable"
41-
)
42-
return False
43-
44-
45-
def _crawl4ai_remote_configured() -> bool:
46-
"""True when a remote Crawl4AI server is configured (CRAWL4AI_API_URL)."""
47-
try:
48-
from content_core.config import get_crawl4ai_api_url
49-
50-
return bool(get_crawl4ai_api_url())
51-
except (ImportError, AttributeError):
52-
return bool(os.environ.get("CRAWL4AI_API_URL"))
53-
except Exception:
54-
logger.opt(exception=True).warning(
55-
"Unexpected error probing Crawl4AI remote config; falling back to env var"
56-
)
57-
return bool(os.environ.get("CRAWL4AI_API_URL"))
58-
59-
60-
def _default_playwright_cache() -> str | None:
61-
"""Playwright's default browser download directory when PLAYWRIGHT_BROWSERS_PATH is unset."""
62-
if sys.platform == "darwin":
63-
return os.path.expanduser("~/Library/Caches/ms-playwright")
64-
if sys.platform == "win32":
65-
local = os.environ.get("LOCALAPPDATA")
66-
return os.path.join(local, "ms-playwright") if local else None
67-
return os.path.expanduser("~/.cache/ms-playwright") # linux and others
68-
69-
70-
def _chromium_browser_present() -> bool:
71-
"""True when a Playwright Chromium browser is installed on disk.
72-
73-
Local Crawl4AI needs both the package AND a Chromium browser. The startup
74-
installer downloads them in separate steps and degrades gracefully, so the
75-
package can be present while the browser download failed — checking the
76-
browser here keeps this endpoint an honest "usable capability" signal.
77-
78-
Playwright installs browsers into PLAYWRIGHT_BROWSERS_PATH (Docker) or, when
79-
that's unset, its per-user default cache (dev). Resolving the path does not
80-
download anything, so we must confirm a chromium build actually exists in
81-
whichever directory applies before reporting local Crawl4AI available.
82-
"""
83-
base = os.environ.get("PLAYWRIGHT_BROWSERS_PATH") or _default_playwright_cache()
84-
if not base or not os.path.isdir(base):
85-
return False
86-
try:
87-
return any("chromium" in name for name in os.listdir(base))
88-
except OSError:
89-
return False
90-
91-
92-
def _crawl4ai_local_ready() -> bool:
93-
"""True when local Crawl4AI can actually render: package installed + Chromium present."""
94-
if importlib.util.find_spec("crawl4ai") is None:
95-
return False
96-
return _chromium_browser_present()
97-
98-
9931
@router.get("", response_model=CapabilitiesResponse)
10032
async def get_capabilities():
10133
"""Report which opt-in extraction runtimes are available in this container."""
102-
crawl4ai_remote = _crawl4ai_remote_configured()
103-
crawl4ai_local = _crawl4ai_local_ready()
34+
crawl4ai_remote = crawl4ai_remote_configured()
35+
crawl4ai_local = crawl4ai_local_ready()
10436
return CapabilitiesResponse(
105-
docling_available=_docling_available(),
37+
docling_available=docling_available(),
10638
crawl4ai_available=crawl4ai_local or crawl4ai_remote,
10739
crawl4ai_remote_configured=crawl4ai_remote,
10840
)

open_notebook/graphs/source.py

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
from open_notebook.domain.notebook import Asset, Source
1616
from open_notebook.domain.transformation import Transformation
1717
from open_notebook.graphs.transformation import graph as transform_graph
18+
from open_notebook.utils.runtime_capabilities import engine_runtime_missing
1819

1920
# Preferred languages for YouTube transcript selection. content-core's own
2021
# default is only ["en", "es", "pt"]; we keep the broader list Open Notebook has
@@ -50,6 +51,27 @@ class TransformationState(TypedDict):
5051
transformation: Transformation
5152

5253

54+
def _usable_engine(engine: str, kind: str) -> str:
55+
"""Return ``engine``, or "auto" when its opt-in runtime is not installed.
56+
57+
The engine choice is persisted in the database; runtime availability comes
58+
from environment flags that are re-evaluated on every boot. A redeploy that
59+
drops OPEN_NOTEBOOK_ENABLE_CRAWL4AI/_DOCLING (or a failed on-demand install)
60+
therefore leaves a stored selection pointing at an absent runtime, and
61+
passing it through fails every extraction with no usable diagnostic. Falling
62+
back to content-core's "auto" chain keeps ingestion working, loudly.
63+
"""
64+
missing_env_var = engine_runtime_missing(engine)
65+
if missing_env_var is None:
66+
return engine
67+
logger.warning(
68+
f"Configured {kind} engine '{engine}' is selected in Content Settings but "
69+
f"its runtime is not available in this container; falling back to 'auto'. "
70+
f"Set {missing_env_var}=true to enable it (see ADR-007)."
71+
)
72+
return "auto"
73+
74+
5375
async def content_process(state: SourceState) -> dict:
5476
content_state: Dict[str, Any] = state["content_state"]
5577

@@ -66,10 +88,12 @@ async def content_process(state: SourceState) -> dict:
6688
try:
6789
settings: ContentSettings = await ContentSettings.get_instance() # type: ignore[assignment]
6890
if settings.default_content_processing_engine_url:
69-
config_kwargs["url_engine"] = settings.default_content_processing_engine_url
91+
config_kwargs["url_engine"] = _usable_engine(
92+
settings.default_content_processing_engine_url, "url"
93+
)
7094
if settings.default_content_processing_engine_doc:
71-
config_kwargs["document_engine"] = (
72-
settings.default_content_processing_engine_doc
95+
config_kwargs["document_engine"] = _usable_engine(
96+
settings.default_content_processing_engine_doc, "document"
7397
)
7498
if settings.docling_ocr is not None:
7599
config_kwargs["docling_ocr"] = settings.docling_ocr
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
"""
2+
Runtime availability probes for the opt-in heavy extraction engines.
3+
4+
Docling and local Crawl4AI are installed on demand at container startup (see
5+
scripts/docker-entrypoint.sh and
6+
docs/7-DEVELOPMENT/decisions/ADR-007-optin-runtimes.md), so availability is a
7+
*runtime* property: it cannot be read off the enable flags, and it does not
8+
survive a redeploy that drops them.
9+
10+
This lives in open_notebook.utils (not api/) because both layers need it: the
11+
capabilities router reports it to the frontend, and the source-processing graph
12+
must not hand content-core an engine whose runtime is absent — the engine
13+
choice is persisted in the database and therefore outlives the environment that
14+
made it available.
15+
"""
16+
17+
import importlib.util
18+
import os
19+
import sys
20+
21+
from loguru import logger
22+
23+
24+
def docling_available() -> bool:
25+
"""True when Docling is installed (its document engine, OCR and image sources work)."""
26+
try:
27+
# content-core's own routing gate — the authoritative signal, not just a spec check.
28+
from content_core.extraction import DOCLING_AVAILABLE
29+
30+
return bool(DOCLING_AVAILABLE)
31+
except (ImportError, AttributeError):
32+
# Content-core absent or its API moved — fall back to a plain spec check.
33+
return importlib.util.find_spec("docling") is not None
34+
except Exception:
35+
# An unexpected failure shouldn't be silently masked as "unavailable".
36+
logger.opt(exception=True).warning(
37+
"Unexpected error probing Docling availability; reporting unavailable"
38+
)
39+
return False
40+
41+
42+
def crawl4ai_remote_configured() -> bool:
43+
"""True when a remote Crawl4AI server is configured (CRAWL4AI_API_URL)."""
44+
try:
45+
from content_core.config import get_crawl4ai_api_url
46+
47+
return bool(get_crawl4ai_api_url())
48+
except (ImportError, AttributeError):
49+
return bool(os.environ.get("CRAWL4AI_API_URL"))
50+
except Exception:
51+
logger.opt(exception=True).warning(
52+
"Unexpected error probing Crawl4AI remote config; falling back to env var"
53+
)
54+
return bool(os.environ.get("CRAWL4AI_API_URL"))
55+
56+
57+
def _default_playwright_cache() -> str | None:
58+
"""Playwright's default browser download directory when PLAYWRIGHT_BROWSERS_PATH is unset."""
59+
if sys.platform == "darwin":
60+
return os.path.expanduser("~/Library/Caches/ms-playwright")
61+
if sys.platform == "win32":
62+
local = os.environ.get("LOCALAPPDATA")
63+
return os.path.join(local, "ms-playwright") if local else None
64+
return os.path.expanduser("~/.cache/ms-playwright") # linux and others
65+
66+
67+
def _chromium_browser_present() -> bool:
68+
"""True when a Playwright Chromium browser is installed on disk.
69+
70+
Local Crawl4AI needs both the package AND a Chromium browser. The startup
71+
installer downloads them in separate steps and degrades gracefully, so the
72+
package can be present while the browser download failed — checking the
73+
browser here keeps this an honest "usable capability" signal.
74+
75+
Playwright installs browsers into PLAYWRIGHT_BROWSERS_PATH (Docker) or, when
76+
that's unset, its per-user default cache (dev). Resolving the path does not
77+
download anything, so we must confirm a chromium build actually exists in
78+
whichever directory applies before reporting local Crawl4AI available.
79+
"""
80+
base = os.environ.get("PLAYWRIGHT_BROWSERS_PATH") or _default_playwright_cache()
81+
if not base or not os.path.isdir(base):
82+
return False
83+
try:
84+
return any("chromium" in name for name in os.listdir(base))
85+
except OSError:
86+
return False
87+
88+
89+
def crawl4ai_local_ready() -> bool:
90+
"""True when local Crawl4AI can actually render: package installed + Chromium present."""
91+
if importlib.util.find_spec("crawl4ai") is None:
92+
return False
93+
return _chromium_browser_present()
94+
95+
96+
def crawl4ai_available() -> bool:
97+
"""True when Crawl4AI can run at all — locally installed or offloaded to a server."""
98+
return crawl4ai_local_ready() or crawl4ai_remote_configured()
99+
100+
101+
# Engine name -> (availability probe, env var that enables it). Engines absent
102+
# from this map need no runtime and are always usable.
103+
_ENGINE_RUNTIMES: dict[str, tuple[str, str]] = {
104+
"crawl4ai": ("crawl4ai_available", "OPEN_NOTEBOOK_ENABLE_CRAWL4AI"),
105+
"docling": ("docling_available", "OPEN_NOTEBOOK_ENABLE_DOCLING"),
106+
}
107+
108+
109+
def engine_runtime_missing(engine: str | None) -> str | None:
110+
"""Return the env var that would enable ``engine``, or None if it is usable.
111+
112+
Used to avoid passing content-core an engine whose runtime is absent, which
113+
fails extraction outright with no indication of the real cause.
114+
"""
115+
if not engine:
116+
return None
117+
entry = _ENGINE_RUNTIMES.get(engine.strip().lower())
118+
if entry is None:
119+
return None # Engine needs no opt-in runtime (auto/simple/firecrawl/jina).
120+
probe_name, env_var = entry
121+
probe = globals()[probe_name]
122+
return None if probe() else env_var

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "open-notebook"
3-
version = "1.13.0"
3+
version = "1.14.0"
44
description = "An open source implementation of a research assistant, inspired by Google Notebook LM"
55
authors = [
66
{name = "Luis Novo", email = "lfnovo@gmail.com"}

0 commit comments

Comments
 (0)