[DOCS] Tools merged between releases ghost in stable add-on docs until next release

[kingpanther13]

Where is the issue?

Tool documentation (homeassistant-addon/DOCS.md, README.md, site/src/data/tools.json)

What's wrong or unclear?

Tools added to src/ha_mcp/tools/ between biweekly stable releases appear in homeassistant-addon/DOCS.md (and README.md and site/src/data/tools.json) on the next push to master, not on the next stable release. Because HA Supervisor refreshes addon-store metadata directly from the GitHub repository at its normal interval, stable add-on users see tools listed in their "Documentation" tab that don't yet exist in the runtime image they have installed. Calling those tools returns a not-found error from the running MCP server.

Concrete current example

The last stable release was v7.2.0 on 2026-04-01 (4b0be35e chore(addon): publish version 7.2.0). PR #876 merged on 2026-04-05 (commit 34a83b4b), four days after the cut, adding ha_remove_entity. As of today, on upstream/master:

• homeassistant-addon/DOCS.md line 379 lists ha_remove_entity (regenerated by sync-tool-docs.yml post-merge).
• ghcr.io/homeassistant-ai/ha-mcp-addon-{arch}:7.2.0, the image installed on every stable HAOS user, contains the pre-feat: add ha_remove_entity tool (closes #874) #876 src/ snapshot — ha_remove_entity is not registered at runtime.

Net effect on a stable user's machine: ha_remove_entity is documented in the add-on's Documentation tab but absent from the live tool list and uncallable. Same pattern applies to every tool-only PR that lands in the gap between releases.

Mechanism

Two refresh cycles run on completely different schedules:

Source	Refresh cadence	Workflow
Add-on metadata (config.yaml, DOCS.md, README.md, translations/) read by HA Supervisor from the repo URL	Minutes after every push to master (Supervisor's normal store refresh)	sync-tool-docs.yml regenerates homeassistant-addon/DOCS.md via extract_tools.py whenever src/ha_mcp/tools/ changes
Add-on runtime image (ghcr.io/.../ha-mcp-addon-{arch}:<version>)	Biweekly Wed 10:00 UTC + manual triggers	addon-publish.yml, called from semver-release.yml

extract_tools.py reads the current src/ tree at every invocation and writes the result to homeassistant-addon/DOCS.md unconditionally. There is no concept of "shipped vs upcoming" in the script today.

How should it be fixed?

Teach extract_tools.py to compare the current src/ha_mcp/tools/ tree against the last stable git tag (git describe --tags --abbrev=0 --match 'v[0-9]*' --exclude '*dev*') and mark any tool present now but absent at that tag as upcoming. Render the marker inline in generate_docs_section and generate_readme_table, plus a one-line explanatory note above the tool list when at least one upcoming tool exists. The upcoming flag is computed dynamically — when semver-release.yml creates the next v7.x.x tag, the comparison baseline moves forward automatically and previously-flagged tools stop being marked. No release-time script needed, no PR-author action needed.

Sketch of the change

1. New helpers in extract_tools.py:
   • _last_stable_tag() — runs git describe, returns tag or None
   • _tool_names_at(tag) — runs git ls-tree + git show + AST-parses each tool file at that tag, returns the set of tool names
2. extract_tools() populates t["upcoming"] on each tool dict
3. generate_docs_section and generate_readme_table append a *(coming next release)* marker when upcoming is true (and beta is not also set, so the markers don't double-up)
4. CI: sync-tool-docs.yml and pr.yml need fetch-depth: 0 + fetch-tags: true on their actions/checkout steps so git describe can find tags
5. Tests in tests/src/unit/ for the new behavior

Total ballpark: ~60-80 lines of Python in extract_tools.py, two one-line CI workflow tweaks, plus tests.

Edge cases handled by the design

• No stable tag exists yet (fresh repo): _last_stable_tag() returns None, no tool gets flagged, script falls back to current behavior.
• Shallow CI checkout: addressed by the fetch-depth: 0 + fetch-tags: true workflow change. Without it, every tool would be marked upcoming, which would be visibly wrong on master and caught immediately.
• git unavailable in the run environment: subprocess.CalledProcessError/FileNotFoundError caught, fall back to no marking.
• Tool renamed between releases: it'd appear as both removed (old name) and upcoming (new name). Acceptable — rare event, and the docs accurately reflect the rename.

Additional context

Discovered while planning PR #942's beta-toggle work for ha_config_set_yaml. PR #942 will introduce a separate static beta tag mechanism for tools that are intentionally dev-channel-only behind a feature flag (different problem, different lifecycle: beta tools may stay beta indefinitely; upcoming tools auto-clear at the next release). The two mechanisms are designed to compose — a tool can be both beta and upcoming, in which case the beta marker takes precedence to avoid label noise. This issue covers the upcoming half only.

[github-actions]

🔍 Analysis

The issue correctly identifies a "ghosting" effect in the Home Assistant add-on documentation. Currently, when a new tool is merged into the master branch, the sync-tool-docs.yml workflow immediately regenerates the documentation. However, the stable add-on image is only updated biweekly. This leads to a mismatch where users see documentation for tools that do not yet exist in their installed version of the MCP server, causing confusion and "not found" errors.

The root cause is that extract_tools.py is "release-unaware"—it treats every tool currently in the source tree as "shipped." The proposed solution is to teach the script to distinguish between tools that existed in the last stable release and those added since then, marking the latter as "upcoming."

This is a valid documentation and lifecycle bug. The suggested fix is elegant because it is dynamic: as soon as a new stable tag is created, the "upcoming" markers for tools in that release will automatically disappear in the next docs sync.

🛠️ Suggested Steps

1. Update scripts/extract_tools.py:
   • Implement _last_stable_tag() using git describe.
   • Implement _tool_names_at(tag) to parse the tool registry at the baseline tag using git ls-tree and git show.
   • Update extract_tools() to flag tools not present in the baseline.
   • Update generate_docs_section and generate_readme_table to append the *(coming next release)* marker.
2. Modify CI Workflows:
   • Update .github/workflows/sync-tool-docs.yml and .github/workflows/pr.yml to set fetch-depth: 0 and fetch-tags: true in the actions/checkout step. This ensures git describe has access to the tag history.
3. Verify Documentation:
   • Run python scripts/extract_tools.py locally and verify that new tools (like ha_remove_entity if it's the latest) are correctly marked.

📊 Technical Analysis (for maintainers)

Root Cause

The extract_tools.py script performs a static analysis of the current filesystem state of src/ha_mcp/tools/ and writes the result directly to DOCS.md. It lacks a comparison baseline to identify tools that have been merged but not yet released in a stable build.

Affected Code

• scripts/extract_tools.py - Needs logic to fetch last stable tag and compare toolsets.
• .github/workflows/sync-tool-docs.yml - Needs actions/checkout update to fetch tags.
• .github/workflows/pr.yml - Needs actions/checkout update for tag-aware validation.

Proposed Fix

--- a/scripts/extract_tools.py
+++ b/scripts/extract_tools.py
@@ -10,6 +10,7 @@
 import argparse
 import ast
 import json
 import re
+import subprocess
 import sys
 from pathlib import Path
@@ -28,6 +29,28 @@
 
+def _last_stable_tag() -> str | None:
+    """Get the latest stable version tag (vX.Y.Z)."""
+    try:
+        cmd = ["git", "describe", "--tags", "--abbrev=0", "--match", "v[0-9]*", "--exclude", "*dev*"]
+        return subprocess.check_output(cmd, stderr=subprocess.DEVNULL, text=True).strip()
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return None
+
+def _tool_names_at(tag: str) -> set[str]:
+    """Identify all tool names present in the codebase at a specific git tag."""
+    names = set()
+    try:
+        cmd = ["git", "ls-tree", "-r", "--name-only", tag, "src/ha_mcp/tools/"]
+        files = subprocess.check_output(cmd, stderr=subprocess.DEVNULL, text=True).splitlines()
+        for f in files:
+            if not (f.endswith(".py") and ("tools_" in f or "backup.py" in f)):
+                continue
+            content = subprocess.check_output(["git", "show", f"{tag}:{f}"], stderr=subprocess.DEVNULL, text=True)
+            # Simplified AST parse to find tool names
+            tree = ast.parse(content)
+            for node in ast.walk(tree):
+                if isinstance(node, ast.AsyncFunctionDef) and node.name.startswith("ha_"):
+                    names.add(node.name)
+    except Exception:
+        pass
+    return names
+
 def extract_tools() -> list[dict]:
     """Extract all tool metadata from source files via AST parsing."""
     tools = []
+    stable_tag = _last_stable_tag()
+    released_tools = _tool_names_at(stable_tag) if stable_tag else set()
 
     for f in TOOL_FILES:
@@ -130,6 +153,7 @@
                 "tags": sorted(tags),
                 "source_file": f.name,
+                "upcoming": tool_name not in released_tools if released_tools else False,
             })

(Note: Similar updates needed in generate_docs_section and generate_readme_table to render the upcoming flag).

Related Issues/PRs

• feat: add ha_remove_entity tool (closes #874) #876 - Merged ha_remove_entity, which is currently ghosting in docs.
• refactor(addon): move ha_config_set_yaml to dev channel only (breaking — stable toggle removed, set ENABLE_YAML_CONFIG_EDITING or use dev add-on) #942 - Planned beta-toggle work which this change is designed to complement.

---

🤖 Automated analysis by Issue Bot