Skip to content

Commit 76708ad

Browse files
feat: expose the in-process server toolset as a Home Assistant LLM API (#1745) (#1782)
* feat: expose the in-process server toolset as a Home Assistant LLM API (#1745) While the in-process server entry is running, its full toolset is registered as an LLM API (homeassistant.helpers.llm), so any Home Assistant conversation agent (OpenAI, Google, Ollama, Anthropic, ...) can select it under "Control Home Assistant" and drive ha-mcp from the Assist chat UI, the companion apps, and voice satellites - the LLM-API direction settled in the issue discussion. The API talks MCP to the server over loopback streamable HTTP, mirroring HA core's own mcp integration: tools are listed fresh each conversation turn (runtime-registered custom tools included), schemas converted via voluptuous_openapi, and each tool call opens a stateless loopback session. The mcp client SDK ships with the runtime-installed server package, so its imports stay lazy; a build without it skips the feature with a warning instead of failing the running server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: catch unwrapped httpx and McpError transport failures in the LLM API Bare httpx errors and protocol-level McpError can escape a session call without the ExceptionGroup wrapper, bypassing the transport-error map and crashing the conversation turn with a raw traceback (Gemini review finding). The except target is now built at exception time - httpx and the mcp SDK arrive with the runtime-installed server package, so they cannot be named in a module-level constant. Mirrors HA core's mcp integration, which catches both the same way. Genuine bugs (ValueError, TypeError, ...) still propagate; regression tests added for all three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: propagate group-wrapped bugs and harden LLM API registration containment Multi-agent review findings on the LLM API: - except ExceptionGroup treated ANY group as a transport failure, but the SDK's anyio task groups wrap in-session bugs (TypeError, KeyError, ...) identically - a genuine bug was relabeled as "could not reach the server" and lost its loud traceback. A group now maps to HomeAssistantError only when every leaf (nested groups included) is a transport error; otherwise it propagates. - async_register_llm_api documented "never raises" but only contained ImportError and HomeAssistantError; anything else leaked into the bring-up's outer except Exception, which tears the ALREADY-RUNNING server down and files a "start" repair issue for a cosmetic failure. The whole registration body is now contained (warn + skip), cancellation still propagates. - HaMcpTool.parameters typed vol.Schema (matches the base contract and HA core's mcp integration reference). Tests: registration-failure containment (both exception classes), group-wrapped/mixed/nested-group bug propagation on both call paths, and the call-path timeout mapping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * test: prove the LLM API e2e against the real server and schema catalog Review roll-in (no follow-up-PR policy): - Embedded e2e now drives the exact client stack llm_api.py uses — real mcp SDK streamable-HTTP session through the webhook, initialize instructions non-empty (the api_prompt source), tools/list, and convert_to_voluptuous over EVERY real tool schema asserting ZERO failures. At runtime an unconvertible schema is skipped per-tool with only a warning, so a systemic voluptuous_openapi incompatibility would silently shrink the toolset; this fails loudly instead. Plus a real call_tool dumped the way HaMcpTool.async_call returns it, and a home-assistant.log assertion that the bring-up actually registered the LLM API inside real HA. voluptuous-openapi added to the dev group for this (HA core provides it at runtime; unit tests stub it). - Unit test for the pre-rename SDK fallback: fakes the transport module at the sys.modules level so _mcp_session's real import selection runs and wires the deprecated streamablehttp_client name identically. Deliberately NOT added: an aggregate "registered X of N tools" warning — every skipped tool is already individually logged with name + traceback, so the aggregate line would add noise without new signal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: raise component log level in the embedded e2e seed and bump skip ceilings The LLM-API registration proof asserts on an INFO line, but the seeded e2e config writes only WARNING+ to home-assistant.log (CI-observed) - a scoped logger block in the container's private config copy surfaces just this component's INFO lines. The two new container_only tests also add exactly +2 marker-skips on each of the haos / haos_inaddon / embedded / haos_embedded lanes; ceilings bumped per the smoke test's own protocol for intentional marker-gated additions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * chore: bump custom component to 1.0.5 for the LLM API feature Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * feat: add an option to enable/disable the conversation-agent LLM API Owner request from live testing: the toolset's LLM API registration is now gated on a server-entry option (default on), with a usage-guide link in the option description. Turning it off removes the API from every conversation agent's selector on the next reload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: inject the LLM API docs link as a description placeholder hassfest forbids literal URLs in strings.json - the options form now fills {llm_api_docs_url} from description_placeholders. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * feat: per-tool LLM API exposure with tool-search mode and settings UI control Owner-directed exposure control for the conversation-agent LLM API: - Server: new llm_exposure module stamps every tools/list entry with _meta.ha_mcp = {llm_api_exposed, pinned}, read live from tool_config's new llm_api overrides map — deny-by-default for beta-tagged tools, ha_dev_* tools, and ha_restart/ha_reload_core/ha_manage_backup. The settings UI grows a fourth per-tool toggle (LLM API) that applies on the agent's next message, no restart; regular MCP clients are untouched (stamp is additive metadata). - Component: llm_api_exposure option — tool_search (default) registers a compact API (pinned tools + ha_search_tools/ha_call_tool meta-tools), full mirrors the whole exposed catalog, both registers the two side by side (per-agent choice via HA's own selector; one server serves all of it). Both modes filter on the stamp; ha_call_tool re-checks exposure at call time and answers a hidden name exactly like a nonexistent one, so hiding cannot be skirted and existence never leaks. Unstamped (older) servers get a conservative built-in deny-list. - Security parity: global disable stays enforced by the server for all clients at list AND call time; policy gate / read-only middleware apply to forwarded agent calls exactly as to any MCP client; e2e asserts ha_restart is present on the raw MCP surface while stamped hidden. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: keep the 1815 toggle placement contract after the merge The startup-notification and sidebar-panel toggles stay immediately after enable_webhook (their test pins that order); the LLM API pair follows them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: restart_required compares default-pinned-padded states The GET handler pads its states response with the default-pinned tools; the JS echoes that padded map back, so an unpadded compare flagged every first save as a states change (live-found while validating the LLM-API toggle). Pad both sides of the compare; LLM-API-exposure-only saves now correctly report restart_required=false. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: hand HA's shared httpx client to the MCP SDK sessions Without it the SDK builds its own httpx client per session, whose SSL setup loads the CA bundle synchronously inside HA's event loop - HA's blocking-call monitor flagged the exact line during live voice-agent testing. HA core's mcp integration passes a hass-managed client for the same reason; the pre-rename SDK fallback has no http_client kwarg and keeps the old behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * test: forwarder assertion includes hass after the shared-client change Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: harden exposure fail-direction, stub beta rendering, and mode default Second multi-agent review round, all findings independently verified: - LlmExposureMiddleware now serves the last-known-good settings when a read fails instead of caching pure defaults - defaults would re-EXPOSE tools the user explicitly hid from agents (fail-open on an exposure control); pure defaults only apply when no read ever succeeded. Plus a non-dict guard on the tool_config payload, and the module docstring no longer overstates 'live on every list' (2s coalescing cache). - Settings UI stub rows for feature-gated tools now render the LLM API toggle hidden-by-default: the previous or-fallback was dead code (stub tags are never empty), so the 7 beta stubs rendered as exposed while the real stamp says hidden. - HaMcpLlmApi.mode defaults to tool_search and unknown modes fall through to the compact shape - the previous full-catalog default made an omitted mode maximally exposed, opposite the option default. - HaMcpTool forwards self.name rather than the (equal) dispatch name. Tests: JSDOM coverage for the LLM API toggle (render, override capture in the POSTed body, live-apply vs restart-required messaging); GET llm_api map incl. the stub-beta default; POST non-dict llm_api 400; TTL expiry; serve-stale-on-read-failure; omitted/unknown mode defaults; and a META-key equality test that mechanically enforces the cross-module stamp contract the runtime import ban forces us to duplicate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv * fix: use from-import for settings_ui module in stub-beta test CodeQL quality gate: the module was imported with both 'import' and 'import from' styles in the same file; match the file's existing 'from ha_mcp import settings_ui as ...' pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XH65D5pHe6k4EDLetdPqDv --------- Co-authored-by: kingpanther13 <kingpanther13@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 753a155 commit 76708ad

23 files changed

Lines changed: 2805 additions & 17 deletions

custom_components/ha_mcp_tools/config_flow.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@
4848
DEFAULT_AUTO_UPDATE,
4949
DEFAULT_BIND_HOST,
5050
DEFAULT_CHANNEL,
51+
DEFAULT_ENABLE_LLM_API,
52+
DEFAULT_LLM_API_EXPOSURE,
5153
DEFAULT_LOOPBACK_URL,
5254
DEFAULT_PIP_SPEC,
5355
DEFAULT_SERVER_PORT,
@@ -56,13 +58,19 @@
5658
DOMAIN,
5759
ENTRY_TYPE_SERVER,
5860
ENTRY_TYPE_TOOLS,
61+
EXPOSURE_BOTH,
62+
EXPOSURE_FULL,
63+
EXPOSURE_TOOL_SEARCH,
64+
LLM_API_DOCS_URL,
5965
OPT_AUTO_UPDATE,
6066
OPT_BIND_HOST,
6167
OPT_CHANNEL,
68+
OPT_ENABLE_LLM_API,
6269
OPT_ENABLE_SIDEBAR_PANEL,
6370
OPT_ENABLE_STARTUP_NOTIFICATION,
6471
OPT_ENABLE_WEBHOOK,
6572
OPT_EXTERNAL_URL,
73+
OPT_LLM_API_EXPOSURE,
6674
OPT_PIP_SPEC,
6775
OPT_REGENERATE_SECRETS,
6876
OPT_SECRET_PATH_OVERRIDE,
@@ -277,6 +285,22 @@ async def async_step_init(
277285
OPT_ENABLE_SIDEBAR_PANEL,
278286
default=bool(opts.get(OPT_ENABLE_SIDEBAR_PANEL, True)),
279287
): bool,
288+
vol.Required(
289+
OPT_ENABLE_LLM_API,
290+
default=bool(opts.get(OPT_ENABLE_LLM_API, DEFAULT_ENABLE_LLM_API)),
291+
): bool,
292+
vol.Required(
293+
OPT_LLM_API_EXPOSURE,
294+
default=str(
295+
opts.get(OPT_LLM_API_EXPOSURE, DEFAULT_LLM_API_EXPOSURE)
296+
),
297+
): SelectSelector(
298+
SelectSelectorConfig(
299+
options=[EXPOSURE_TOOL_SEARCH, EXPOSURE_FULL, EXPOSURE_BOTH],
300+
translation_key="llm_api_exposure",
301+
mode=SelectSelectorMode.DROPDOWN,
302+
)
303+
),
280304
vol.Optional(
281305
OPT_EXTERNAL_URL,
282306
default=opts.get(OPT_EXTERNAL_URL, ""),
@@ -312,6 +336,7 @@ async def async_step_init(
312336
description_placeholders={
313337
"versions": await self._versions_hint(),
314338
"connect_url": self._connect_url_hint(),
339+
"llm_api_docs_url": LLM_API_DOCS_URL,
315340
"panel_hint": panel_hint,
316341
},
317342
)

custom_components/ha_mcp_tools/const.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,23 @@ def channel_for_dist(dist: str) -> str:
288288
# server through Home Assistant; only the direct server port (+ the
289289
# admin-only sidebar panel, which proxies over loopback) remains.
290290
OPT_ENABLE_WEBHOOK = "enable_webhook"
291+
# Conversation-agent LLM API (#1745): when False, the toolset is not
292+
# registered as a Home Assistant LLM API, so it never appears in any
293+
# conversation agent's "Control Home Assistant" selector. On by default —
294+
# registering the API only makes it selectable; nothing is exposed until a
295+
# user picks it on an agent.
296+
OPT_ENABLE_LLM_API = "enable_llm_api"
297+
DEFAULT_ENABLE_LLM_API = True
298+
# Which exposure shape(s) the LLM API offers to conversation agents:
299+
# ``tool_search`` (default) registers a compact API — pinned tools plus
300+
# search/execute meta-tools — the shape context-limited models need; ``full``
301+
# registers the whole exposed catalog as one API; ``both`` registers the two
302+
# side by side so the choice is made per agent in HA's own selector.
303+
OPT_LLM_API_EXPOSURE = "llm_api_exposure"
304+
EXPOSURE_TOOL_SEARCH = "tool_search"
305+
EXPOSURE_FULL = "full"
306+
EXPOSURE_BOTH = "both"
307+
DEFAULT_LLM_API_EXPOSURE = EXPOSURE_TOOL_SEARCH
291308
# When False, the persistent notification created on every server bring-up is
292309
# suppressed; the connect URLs still reach the admin-only Home Assistant log.
293310
OPT_ENABLE_STARTUP_NOTIFICATION = "enable_startup_notification"
@@ -335,6 +352,9 @@ def channel_for_dist(dist: str) -> str:
335352
# finding on #1760). Bring-up pops it: notification on success, silent drop on
336353
# failure (the package/start repair issues cover that path).
337354
DATA_PENDING_UPDATE_NOTIFY = "pending_update_notify"
355+
# Unregister callback for the conversation-agent LLM API (#1745), stored by
356+
# the bring-up success path and invoked (idempotently) by teardown.
357+
DATA_LLM_API_UNSUB = "llm_api_unsub"
338358

339359
# Webhook auth modes (mirrors the webhook-proxy add-on's default posture).
340360
WEBHOOK_AUTH_NONE = "none" # secret webhook URL is the shared secret (default)
@@ -379,6 +399,15 @@ def channel_for_dist(dist: str) -> str:
379399
"?owner=homeassistant-ai&repository=ha-mcp-integration&category=integration"
380400
)
381401

402+
# Usage guide for the conversation-agent LLM API option (#1745). Injected into
403+
# the options form as a description placeholder — hassfest forbids literal
404+
# URLs inside strings.json.
405+
LLM_API_DOCS_URL = (
406+
"https://github.com/homeassistant-ai/ha-mcp/blob/master/docs/"
407+
"in-process-server.md"
408+
"#chat-with-the-toolset-from-home-assistant-conversation-agents--voice"
409+
)
410+
382411
# Repair-issue ids surfaced when server bring-up fails.
383412
ISSUE_PACKAGE_FAILED = "server_package_install_failed"
384413
ISSUE_START_FAILED = "server_start_failed"

custom_components/ha_mcp_tools/embedded_setup.py

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
DATA_WEBHOOK_ID,
4040
DEFAULT_AUTO_UPDATE,
4141
DEFAULT_BIND_HOST,
42+
DEFAULT_ENABLE_LLM_API,
4243
DEFAULT_PIP_SPEC,
4344
DEFAULT_SERVER_PORT,
4445
DOMAIN,
@@ -49,6 +50,7 @@
4950
ISSUE_UPDATE_HELD,
5051
OPT_AUTO_UPDATE,
5152
OPT_BIND_HOST,
53+
OPT_ENABLE_LLM_API,
5254
OPT_ENABLE_SIDEBAR_PANEL,
5355
OPT_ENABLE_STARTUP_NOTIFICATION,
5456
OPT_ENABLE_WEBHOOK,
@@ -60,6 +62,7 @@
6062
channel_for_dist,
6163
)
6264
from .embedded_server import EmbeddedServerError, EmbeddedServerManager
65+
from .llm_api import async_register_llm_api, async_unregister_llm_api
6366
from .mcp_webhook import async_register_webhook, async_unregister_webhook
6467

6568
if TYPE_CHECKING:
@@ -125,6 +128,18 @@ async def async_bring_up_server(hass: HomeAssistant, entry: ConfigEntry) -> None
125128
"(direct port + sidebar panel)"
126129
)
127130
_surface_connect_urls(hass, entry, auth_mode, webhook_enabled=webhook_enabled)
131+
# Conversation-agent LLM API (#1745), gated on its option (default on).
132+
# Advisory: registration failures are contained inside (logged, feature
133+
# absent) — the running server must never be taken down by them.
134+
if bool(entry.options.get(OPT_ENABLE_LLM_API, DEFAULT_ENABLE_LLM_API)):
135+
await async_register_llm_api(
136+
hass, entry, port=manager.port, secret_path=secret_path
137+
)
138+
else:
139+
_LOGGER.info(
140+
"Conversation-agent LLM API disabled by option - the toolset "
141+
"will not be offered to Home Assistant conversation agents"
142+
)
128143
await _async_finish_update_cycle(hass)
129144
except asyncio.CancelledError:
130145
# Unloaded mid-bring-up: undo whatever partial state exists, then let the
@@ -153,12 +168,14 @@ async def async_bring_up_server(hass: HomeAssistant, entry: ConfigEntry) -> None
153168

154169

155170
async def async_teardown_server(hass: HomeAssistant) -> None:
156-
"""Unregister the webhook and stop the server thread (reload-safe, idempotent).
171+
"""Unregister the LLM API + webhook and stop the server thread (reload-safe,
172+
idempotent).
157173
158174
Does NOT revoke the provisioned token — a reload must keep it. The ha_auth
159175
discovery views stay bound (aiohttp can't unregister them until HA restarts);
160176
they 404 while the entry is not live.
161177
"""
178+
async_unregister_llm_api(hass)
162179
await async_unregister_webhook(hass)
163180
manager = hass.data.get(DOMAIN, {}).pop(DATA_MANAGER, None)
164181
if isinstance(manager, EmbeddedServerManager):

0 commit comments

Comments
 (0)