feat: harden ha_config_set_yaml description and require justification

LLM agents routinely reach for ha_config_set_yaml to create trigger-based
template sensors even though ha_set_config_entry_helper (domain=template)
supports them natively. The previous description listed
ha_config_set_helper as the preferred alternative for template sensors,
but ha_config_set_helper's helper_type enum does not include 'template' —
so an agent that tried to follow the guidance would hit a schema error
and then fall back to raw YAML editing.

This change:

- Reframes the docstring and tool title as an escape hatch. The first
  sentence now warns instead of welcoming, and the dedicated-tool
  preference list is the first thing a model sees.
- Points template sensors (state-based AND trigger-based) at the
  correct tool, ha_set_config_entry_helper with helper_type='template',
  which does support triggers via the config entry flow.
- Hardens the yaml_path parameter description with per-key warnings, so
  the nudge survives context compression even when the full docstring
  is trimmed by the client.
- Adds a required justification parameter (mirroring the pattern from
  ha_manage_custom_tool in homeassistant-ai#854). Justification is validated non-empty,
  logged, and exists purely as friction — the goal is to make an agent
  pause and articulate why no dedicated tool fits before reaching for
  the escape hatch.

Ref: discussion homeassistant-ai#936

Author: kingpanther13

homeassistant-addon/DOCS.md

@@ -423,7 +423,7 @@ The add-on provides 86+ MCP tools for controlling Home Assistant:
 - `ha_backup_create` — Create a fast Home Assistant backup (local only).
 - `ha_backup_restore` — Restore Home Assistant from a backup (LAST RESORT - use with extreme caution).
 - `ha_check_config` — Check Home Assistant configuration for errors.
-- `ha_config_set_yaml` — Add, replace, or remove a top-level key in configuration.yaml or package files.
+- `ha_config_set_yaml` — ESCAPE HATCH — raw YAML edit. Use only when NO dedicated tool fits.
 - `ha_get_system_health` — Get Home Assistant system health, including Zigbee (ZHA) and Z-Wave JS network diagnostics.
 - `ha_get_updates` — Get update information - list all updates or get details for a specific one.
 - `ha_reload_core` — Reload Home Assistant configuration without full restart.

site/src/data/tools.json

@@ -2414,16 +2414,20 @@
   },
   {
     "name": "ha_config_set_yaml",
-    "title": "Set YAML Config",
-    "description": "Add, replace, or remove a top-level key in configuration.yaml or package files.\n\nIMPORTANT: Only use when NO UI or API alternative exists. Prefer:\n- Template sensors -> ha_config_set_helper (Template Helper)\n- Automations -> ha_config_set_automation\n- Scripts -> ha_config_set_script\n- Input helpers -> ha_config_set_helper\n- Scenes -> ha_config_set_scene\n\nThis tool is for YAML-only features with no UI/API path (e.g.,\ncommand_line sensors, platform-based MQTT sensors in YAML, rest\nsensors defined in packages).\n\nSafeguards: file backup, YAML validation, top-level key whitelist,\npath traversal blocking, post-edit config check.\n\nIMPORTANT: Check 'post_action' in the response. Most keys require\na full HA restart ('restart_required'). Only template, mqtt, and\ngroup support reload ('reload_available' with 'reload_service').\n\nPreserves YAML comments on sibling keys, file-level comments,\nand Home Assistant tags (!include, !secret, etc.). The 'replace' action\nsubstitutes the subtree as-is, so comments from the old subtree\ndo not carry over.",
+    "title": "Raw YAML Config Edit (Escape Hatch)",
+    "description": "ESCAPE HATCH — raw YAML edit. Use only when NO dedicated tool fits.\n\nThis tool is the WRONG answer for almost everything. Before calling it,\nconfirm that NONE of these apply:\n\n- Template sensors (state-based OR trigger-based) ->\n  ha_set_config_entry_helper with domain='template'. The Template\n  config-flow helper supports triggers, availability, attributes, device\n  class, unit of measurement, and state templates since HA 2024.x. It is\n  the correct path even for complex trigger-based sensors. Do NOT edit\n  the 'template:' YAML block for this.\n- Automations -> ha_config_set_automation\n- Scripts -> ha_config_set_script\n- Scenes -> ha_config_set_scene\n- Input helpers (input_boolean, input_number, input_text, input_select,\n  input_datetime, input_button, counter, timer, schedule) ->\n  ha_config_set_helper\n- Groups, min/max, threshold, derivative, statistics, utility_meter\n  entities that can be created as helpers -> ha_config_set_helper\n\nThis tool is intended for YAML-only integrations that have no config-flow\nor API equivalent: command_line sensors, REST sensors defined in\npackages, shell_command entries, platform-based notify services, and\nsimilar edge cases. If you are reaching for this to edit 'template:',\n'automation:', 'script:', 'scene:', or 'input_*:', stop and use the\ndedicated tool instead.\n\nA ``justification`` is REQUIRED on every call and is logged for auditing.\nThe justification must explain why no dedicated tool fits. If you cannot\nwrite a concrete justification, you are using the wrong tool.\n\nSafeguards: file backup, YAML validation, top-level key allowlist,\npath traversal blocking, post-edit config check.\n\nCheck 'post_action' in the response. Most keys require a full HA restart\n('restart_required'). Only template, mqtt, and group support reload\n('reload_available' with 'reload_service').\n\nPreserves YAML comments on sibling keys, file-level comments, and Home\nAssistant tags (!include, !secret, etc.). The 'replace' action\nsubstitutes the subtree as-is, so comments from the old subtree do not\ncarry over.",
     "inputSchema": {
       "properties": {
         "yaml_path": {
-          "type": "Annotated[str, Field(description=\"Top-level YAML key to modify (e.g., 'template', 'sensor', 'input_boolean'). Only whitelisted keys are allowed.\")]"
+          "type": "Annotated[str, Field(description=\"Top-level YAML key to modify. Only a narrow allowlist of YAML-only integration keys is accepted (e.g., 'command_line', 'rest', 'shell_command', 'notify', 'utility_meter'). STOP before using this for 'template' — the Template config-flow helper (ha_set_config_entry_helper with domain='template') supports state AND trigger-based template sensors since HA 2024.x and is the correct path. Do NOT use this tool for automations, scripts, scenes, or input_* helpers — they have dedicated tools (ha_config_set_automation, ha_config_set_script, ha_config_set_scene, ha_config_set_helper).\")]"
         },
         "action": {
           "type": "Annotated[str, Field(description=\"Action to perform: 'add' (insert/merge content under key), 'replace' (overwrite key with new content), or 'remove' (delete the key entirely).\")]"
         },
+        "justification": {
+          "type": "Annotated[str | None, Field(default=None, description='Required. Explain in one or two sentences why no dedicated tool (ha_set_config_entry_helper for templates, ha_config_set_automation, ha_config_set_script, ha_config_set_scene, ha_config_set_helper) can accomplish this task. Logged for auditing. If you cannot write a concrete justification, you probably want a different tool.')]",
+          "default": null
+        },
         "content": {
           "type": "Annotated[str | None, Field(default=None, description=\"YAML content for the value under yaml_path. Required for 'add' and 'replace' actions. Must be valid YAML.\")]",
           "default": null

src/ha_mcp/tools/tools_yaml_config.py

@@ -48,7 +48,7 @@ def register_yaml_config_tools(mcp: Any, client: Any, **kwargs: Any) -> None:
         annotations={
             "destructiveHint": True,
             "idempotentHint": False,
-            "title": "Set YAML Config",
+            "title": "Raw YAML Config Edit (Escape Hatch)",
         },
     )
     @log_tool_usage
@@ -57,8 +57,17 @@ async def ha_config_set_yaml(
             str,
             Field(
                 description=(
-                    "Top-level YAML key to modify (e.g., 'template', 'sensor', "
-                    "'input_boolean'). Only whitelisted keys are allowed."
+                    "Top-level YAML key to modify. Only a narrow allowlist of "
+                    "YAML-only integration keys is accepted (e.g., 'command_line', "
+                    "'rest', 'shell_command', 'notify', 'utility_meter'). "
+                    "STOP before using this for 'template' — the Template "
+                    "config-flow helper (ha_set_config_entry_helper with "
+                    "domain='template') supports state AND trigger-based template "
+                    "sensors since HA 2024.x and is the correct path. Do NOT use "
+                    "this tool for automations, scripts, scenes, or input_* "
+                    "helpers — they have dedicated tools "
+                    "(ha_config_set_automation, ha_config_set_script, "
+                    "ha_config_set_scene, ha_config_set_helper)."
                 ),
             ),
         ],
@@ -72,6 +81,20 @@ async def ha_config_set_yaml(
                 ),
             ),
         ],
+        justification: Annotated[
+            str | None,
+            Field(
+                default=None,
+                description=(
+                    "Required. Explain in one or two sentences why no dedicated "
+                    "tool (ha_set_config_entry_helper for templates, "
+                    "ha_config_set_automation, ha_config_set_script, "
+                    "ha_config_set_scene, ha_config_set_helper) can accomplish "
+                    "this task. Logged for auditing. If you cannot write a "
+                    "concrete justification, you probably want a different tool."
+                ),
+            ),
+        ] = None,
         content: Annotated[
             str | None,
             Field(
@@ -103,30 +126,48 @@ async def ha_config_set_yaml(
             ),
         ] = True,
     ) -> dict[str, Any]:
-        """Add, replace, or remove a top-level key in configuration.yaml or package files.
+        """ESCAPE HATCH — raw YAML edit. Use only when NO dedicated tool fits.
+
+        This tool is the WRONG answer for almost everything. Before calling it,
+        confirm that NONE of these apply:
 
-        IMPORTANT: Only use when NO UI or API alternative exists. Prefer:
-        - Template sensors -> ha_config_set_helper (Template Helper)
+        - Template sensors (state-based OR trigger-based) ->
+          ha_set_config_entry_helper with domain='template'. The Template
+          config-flow helper supports triggers, availability, attributes, device
+          class, unit of measurement, and state templates since HA 2024.x. It is
+          the correct path even for complex trigger-based sensors. Do NOT edit
+          the 'template:' YAML block for this.
         - Automations -> ha_config_set_automation
         - Scripts -> ha_config_set_script
-        - Input helpers -> ha_config_set_helper
         - Scenes -> ha_config_set_scene
+        - Input helpers (input_boolean, input_number, input_text, input_select,
+          input_datetime, input_button, counter, timer, schedule) ->
+          ha_config_set_helper
+        - Groups, min/max, threshold, derivative, statistics, utility_meter
+          entities that can be created as helpers -> ha_config_set_helper
 
-        This tool is for YAML-only features with no UI/API path (e.g.,
-        command_line sensors, platform-based MQTT sensors in YAML, rest
-        sensors defined in packages).
+        This tool is intended for YAML-only integrations that have no config-flow
+        or API equivalent: command_line sensors, REST sensors defined in
+        packages, shell_command entries, platform-based notify services, and
+        similar edge cases. If you are reaching for this to edit 'template:',
+        'automation:', 'script:', 'scene:', or 'input_*:', stop and use the
+        dedicated tool instead.
 
-        Safeguards: file backup, YAML validation, top-level key whitelist,
+        A ``justification`` is REQUIRED on every call and is logged for auditing.
+        The justification must explain why no dedicated tool fits. If you cannot
+        write a concrete justification, you are using the wrong tool.
+
+        Safeguards: file backup, YAML validation, top-level key allowlist,
         path traversal blocking, post-edit config check.
 
-        IMPORTANT: Check 'post_action' in the response. Most keys require
-        a full HA restart ('restart_required'). Only template, mqtt, and
-        group support reload ('reload_available' with 'reload_service').
+        Check 'post_action' in the response. Most keys require a full HA restart
+        ('restart_required'). Only template, mqtt, and group support reload
+        ('reload_available' with 'reload_service').
 
-        Preserves YAML comments on sibling keys, file-level comments,
-        and Home Assistant tags (!include, !secret, etc.). The 'replace' action
-        substitutes the subtree as-is, so comments from the old subtree
-        do not carry over.
+        Preserves YAML comments on sibling keys, file-level comments, and Home
+        Assistant tags (!include, !secret, etc.). The 'replace' action
+        substitutes the subtree as-is, so comments from the old subtree do not
+        carry over.
         """
         try:
             # Validate action
@@ -144,6 +185,31 @@ async def ha_config_set_yaml(
                     )
                 )
 
+            # Require a non-empty justification. This is the primary friction
+            # that discourages reaching for this escape hatch — if the caller
+            # cannot articulate why no dedicated tool fits, they should use a
+            # dedicated tool instead. Mirrors the pattern from tools_code.py
+            # (ha_manage_custom_tool).
+            if not justification or not justification.strip():
+                raise_tool_error(
+                    create_error_response(
+                        ErrorCode.VALIDATION_INVALID_PARAMETER,
+                        "justification is required for ha_config_set_yaml",
+                        suggestions=[
+                            "Explain why no dedicated tool can accomplish this "
+                            "task (Template sensors -> "
+                            "ha_set_config_entry_helper with "
+                            "domain='template'; automations -> "
+                            "ha_config_set_automation; scripts -> "
+                            "ha_config_set_script; helpers -> "
+                            "ha_config_set_helper)",
+                            "If yaml_path is 'template', 'automation', "
+                            "'script', 'scene', or 'input_*', use the "
+                            "dedicated tool instead of this escape hatch",
+                        ],
+                    )
+                )
+
             # Validate content is provided for add/replace
             if action in ("add", "replace") and not content:
                 raise_tool_error(
@@ -156,6 +222,14 @@ async def ha_config_set_yaml(
                     )
                 )
 
+            logger.info(
+                "ha_config_set_yaml invoked (yaml_path=%s, action=%s, file=%s) — justification: %s",
+                yaml_path,
+                action,
+                file,
+                justification[:200],
+            )
+
             # Coerce boolean parameter
             backup_bool = coerce_bool_param(backup, "backup", default=True)