feat: server update entity, auto-update notifications, and real HACS release notes (#1760) (#1776)

* feat: surface in-process server updates via an update entity and repair issues (#1760)

The embedded server self-updates from PyPI with only an INFO log line, and
with auto-update off the PyPI poll never ran at all, so users had zero signal
that a server update existed. The component and server stay independently
versioned (no lockstep); this adds the missing visibility:

- ServerVersionCoordinator polls installed + PyPI-latest on the existing 6h
  interval regardless of the auto_update option; only the reload stays gated
  on it (plus a new guard against reloading while bring-up is still in
  flight, since the first refresh now runs shortly after setup).
- New update platform entity per server entry: installed/latest version,
  auto_update reflection, channel-aware release_url, GitHub release notes on
  the stable channel, device sw_version, and a manual Install that works with
  auto-update off via a one-shot pending-install marker the server manager
  pins to and clears after a successful install.
- Persistent notification (no secrets) when an automatic update installs,
  naming old/new versions and linking the release notes.
- Legacy-HACS-source repair issue: installs whose HACS entry still tracks the
  main server repo see 7.x server versions and server release notes in HACS;
  detect that via HACS's registry after HA start and point at the mirror.
  The legacy path keeps working.
- hacs.json HA floor to 2024.11.0 (DataUpdateCoordinator's config_entry
  kwarg exists since 2024.11).
- Component version 0.15.0. Hermetic unit tests for all of the above; unit
  test stubs no longer clobber each other's homeassistant.core module
  (sys.modules setdefault), which full-suite collection order exposed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: generate real component release notes for the HACS mirror (#1760)

Mirror releases carried a one-line stub body, so even clean mirror installs
saw no meaningful changelog in the HACS update dialog. The deploy key can
push git refs but cannot set a release body via the API, so the notes travel
inside an annotated tag message: the sync workflow builds them from commit
subjects touching custom_components/ha_mcp_tools/ between the last two
stable server releases (scripts/build_mirror_release_notes.py) and tags with
--cleanup=verbatim (the -F default strips #-prefixed markdown headings). The
mirror's release-on-tag workflow now lives canonically in this repo under
.github/integration-mirror-workflows/, is synced over like the rest of the
mirror, and uses the tag message as the release body with the old stub as
fallback for lightweight tags.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: document the update entity, auto-update notification, and legacy-repo repair issue

Also fixes the now-stale claim that turning automatic updates off stops the
periodic check — the check keeps running to feed the update entity; only the
automatic install is gated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump the custom component to 1.0.0

Owner decision: the component has shipped the full in-process server since
the installation overhaul and should have left 0.x then; doing it with this
release. Supersedes the 0.15.0 bump earlier on this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: harden the update flow per PR review (marker one-shot, notify-on-success, honest Install)

Five review agents plus Gemini surfaced interlocking gaps in the new update
flow; all confirmed against source and fixed:

- Pending-install marker is consumed on ATTEMPT, not on success: cleared only
  on success, a marker for a failing version re-pinned every later reload --
  including periodic auto-update ones -- to the same broken version, looping
  the failure forever with the server down (critical finding).
- The "server updated" notification no longer fires optimistically right
  after requesting the reload: async_maybe_auto_update leaves a hass.data
  marker and bring-up's success path fires the notification with the
  actually-installed version; failure paths drop it (repair issues cover
  them). The auto-update reload is also guarded now -- a raising reload gets
  an ERROR log instead of dying silently in a background task.
- Bring-up's success path refreshes the version coordinator, so the update
  entity reflects a completed install immediately instead of showing a stale
  "update available" for up to 6 hours.
- update entity's Install now waits for the reloaded entry's bring-up and
  verifies the requested version actually landed, raising into the update UI
  when it did not; previously it reported success at reload time, before the
  pip install had even run.
- async_release_notes logs unexpected payload shapes at warning with
  traceback (debug stays for expected transients); async_install logs before
  wrapping into HomeAssistantError.
- Mirror release-on-tag workflow: lightweight-tag detection now uses the tag
  object type (%(contents) falls through to the commit message, so the old
  emptiness check could never fire), and release-create failures are no
  longer swallowed as "release exists" -- existence is checked explicitly.
- async_maybe_auto_update's info parameter typed ServerVersionInfo | None to
  match its documented and tested None handling; channel<->dist mapping
  consolidated into const.dist_for_channel/channel_for_dist (5 inline copies).
- Stale text swept: const.py's auto-update comment blocks still described the
  old poll-stops-when-off behavior; the legacy-repo repair issue said 0.x for
  a 1.x component.
- Coordinator stored before the bring-up task is created (its success path
  now reads it), and tests for every behavior above (+14).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: kingpanther13 <kingpanther13@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: kingpanther13

.github/integration-mirror-workflows/release-on-tag.yml

@@ -0,0 +1,68 @@
+name: Release on tag
+
+# Canonical copy: this file lives in the main repo (homeassistant-ai/ha-mcp)
+# at .github/integration-mirror-workflows/release-on-tag.yml and is synced
+# into the ha-mcp-integration mirror by sync-integration-mirror.yml. Do not
+# hand-edit it in the mirror -- edits are overwritten on the next sync.
+#
+# The monorepo sync pushes vX.Y.Z tags (deploy keys cannot call the GitHub
+# API to create a release directly, only push git refs -- including
+# annotated tags and, per this same sync workflow, this file). Since the
+# deploy key can't set a release body via the API either, the tag itself
+# carries the body: the sync workflow creates an ANNOTATED tag whose message
+# is the generated component release notes, and this job uses that message
+# verbatim as the release body. Tags predating this change (or any that
+# somehow end up lightweight/empty) fall back to the old generic stub so
+# behavior degrades gracefully instead of shipping a blank release.
+on:
+  push:
+    tags: ["v*"]
+
+permissions:
+  contents: write
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          # A shallow, ref-only checkout of a tag-push event is not
+          # guaranteed to bring down the full annotated tag object (only
+          # the commit it points to) -- fetch full history and tags so the
+          # tag message is actually present locally for the next step.
+          fetch-depth: 0
+          fetch-tags: true
+
+      - name: Extract tag message
+        run: |
+          # A lightweight tag has no message of its own — %(contents) on one
+          # falls through to the pointed-to COMMIT's message, so an emptiness
+          # check cannot detect it (review finding). Check the tag object's
+          # actual type instead: only a real annotated tag ("tag" object)
+          # carries the generated release notes; anything else gets the stub.
+          if [ "$(git cat-file -t "refs/tags/$GITHUB_REF_NAME")" = "tag" ]; then
+            git tag -l --format='%(contents)' "$GITHUB_REF_NAME" > /tmp/notes.md
+          else
+            : > /tmp/notes.md
+          fi
+          if ! grep -q '[^[:space:]]' /tmp/notes.md; then
+            echo "Component snapshot synced from homeassistant-ai/ha-mcp." > /tmp/notes.md
+          fi
+
+      - name: Create release for tag
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          VER="${GITHUB_REF_NAME#v}"
+          # Check existence explicitly instead of `|| echo "release exists"`:
+          # that pattern treated EVERY create failure (auth, rate limit, bad
+          # notes file) as already-exists and reported green with no release
+          # published (review finding). Now only a genuine duplicate no-ops;
+          # a real create failure fails the job visibly.
+          if gh release view "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
+            echo "release exists"
+            exit 0
+          fi
+          gh release create "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" \
+            --title "$VER" -F /tmp/notes.md

.github/workflows/sync-integration-mirror.yml

@@ -25,6 +25,7 @@ on:
       - "README.md"
       - ".github/integration-mirror-readme-prefix.md"
       - "scripts/build_mirror_readme.py"
+      - ".github/integration-mirror-workflows/**"
   workflow_run:
     workflows: ["SemVer Release"]
     types: [completed]
@@ -35,6 +36,12 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
+        with:
+          # The release-notes step needs the full stable-tag history
+          # (git tag -l 'v*') to find the previous and current server
+          # release, not just the default single-commit shallow checkout.
+          fetch-depth: 0
+          fetch-tags: true
 
       - name: Set up mirror push key
         env:
@@ -62,6 +69,13 @@ jobs:
             --prefix .github/integration-mirror-readme-prefix.md \
             --readme README.md \
             --out /tmp/mirror/README.md
+          # Deploy keys can push workflow files (unlike a PAT, they aren't
+          # subject to the `workflow` scope restriction), and the mirror
+          # carries no hand-made changes, so its release-on-tag.yml is
+          # synced from here too rather than hand-edited in the mirror.
+          mkdir -p /tmp/mirror/.github/workflows
+          cp .github/integration-mirror-workflows/release-on-tag.yml \
+            /tmp/mirror/.github/workflows/release-on-tag.yml
 
       - name: Commit and push
         env:
@@ -80,11 +94,18 @@ jobs:
 
       - name: Tag mirror for stable release
         # Tags the mirror with the current component version after a successful
-        # SemVer Release (or a manual re-run). The mirror's own release-on-tag
-        # workflow turns this tag into a GitHub release (the monorepo
-        # GITHUB_TOKEN has no cross-repo API access; the deploy key can only
-        # push). Idempotent: an existing tag no-ops, so a release that did not
-        # change the component version simply re-confirms the current tag.
+        # SemVer Release (or a manual re-run). The tag is ANNOTATED and its
+        # message carries the release notes body: the deploy key can push git
+        # refs (including annotated tags) but has no cross-repo GitHub API
+        # access to set a release body directly, so the mirror's
+        # release-on-tag.yml (synced from
+        # .github/integration-mirror-workflows/release-on-tag.yml) reads the
+        # tag message and uses it as-is. `--cleanup=verbatim` is required:
+        # `git tag -a -F` defaults to `--cleanup=strip`, which silently drops
+        # any line starting with "#" -- including the markdown heading the
+        # notes script emits. Idempotent: an existing tag no-ops, so a
+        # release that did not change the component version simply
+        # re-confirms the current tag.
         if: >-
           github.event_name == 'workflow_dispatch' ||
           (github.event_name == 'workflow_run' &&
@@ -93,10 +114,15 @@ jobs:
           GIT_SSH_COMMAND: ssh -i ~/.ssh/mirror_key -o IdentitiesOnly=yes
         run: |
           VER=$(python3 -c "import json; print(json.load(open('custom_components/ha_mcp_tools/manifest.json'))['version'])")
-          cd /tmp/mirror
-          if git ls-remote --tags origin "refs/tags/v${VER}" | grep -q .; then
+          if git -C /tmp/mirror ls-remote --tags origin "refs/tags/v${VER}" | grep -q .; then
             echo "mirror tag v${VER} exists"
             exit 0
           fi
-          git tag "v${VER}"
+          python3 scripts/build_mirror_release_notes.py \
+            --component-version "$VER" \
+            --out /tmp/mirror_notes.md
+          echo "--- release notes for v${VER} ---"
+          cat /tmp/mirror_notes.md
+          cd /tmp/mirror
+          git tag -a "v${VER}" -F /tmp/mirror_notes.md --cleanup=verbatim
           git push origin "v${VER}"

custom_components/ha_mcp_tools/__init__.py

@@ -1771,6 +1771,16 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
     (issue #1527). A missing ``entry_type`` means ``tools`` so pre-existing
     entries keep working across the component update with no migration.
     """
+    # Lazy import (matches the embedded chain's own convention in
+    # embedded_entry.py): keeps homeassistant.const's import surface out of
+    # every hermetic unit test that merely imports this package, not just the
+    # ones that actually call async_setup_entry.
+    from .install_source_check import async_schedule_install_source_check
+
+    # Runs for BOTH entry types: HACS delivers the component as a whole, not
+    # just the server entry, so a legacy-repo install needs detecting either way.
+    async_schedule_install_source_check(hass)
+
     if entry.data.get(CONF_ENTRY_TYPE) == ENTRY_TYPE_SERVER:
         from .embedded_entry import async_setup_server_entry
 

custom_components/ha_mcp_tools/const.py

@@ -212,12 +212,31 @@
 CHANNEL_DEV = "dev"
 DEFAULT_CHANNEL = CHANNEL_STABLE
 
-# Automatic server-version updates. Both channels are unpinned, so an entry
-# reload / HA restart already reinstalls the newest build; on top of that the
-# component polls PyPI on this interval and reloads the entry when a newer build
-# is published, so a long-running instance picks up releases without a restart.
-# An explicit pip-spec override disables the check, as does turning off the
-# ``auto_update`` option (OPT_AUTO_UPDATE).
+
+def dist_for_channel(channel: str) -> str:
+    """Map a release channel to its PyPI distribution name.
+
+    The channel <-> distribution correspondence is used by the version
+    coordinator, the auto-update notification, and the server manager's pip
+    resolution — one shared mapping so a future third channel cannot be added
+    to some sites and missed in others (review finding on #1760).
+    """
+    return DIST_NAME_DEV if channel == CHANNEL_DEV else DIST_NAME_STABLE
+
+
+def channel_for_dist(dist: str) -> str:
+    """Inverse of :func:`dist_for_channel`."""
+    return CHANNEL_DEV if dist == DIST_NAME_DEV else CHANNEL_STABLE
+
+
+# Interval of the ServerVersionCoordinator's PyPI poll (coordinator.py). The
+# poll itself ALWAYS runs — it feeds the `update` platform entity, which must
+# stay populated even when automatic updates are off (issue #1760). Whether a
+# newer build actually triggers a reload/reinstall is decided separately, per
+# refresh, in embedded_setup.async_maybe_auto_update (gated on OPT_AUTO_UPDATE
+# and on no pip-spec override). Only an explicit pip-spec override skips the
+# PyPI fetch — comparing PyPI-latest against an arbitrary pip spec is
+# meaningless.
 UPDATE_CHECK_INTERVAL = timedelta(hours=6)
 
 # PyPI JSON API for the latest published version of a distribution. ``{dist}``
@@ -227,11 +246,13 @@
 # Options-flow keys (stored in entry.options).
 OPT_CHANNEL = "channel"
 # Automatic server-version updates toggle (default on). When on, the channel is
-# unpinned and auto-updates (force-install on reload/restart + the periodic
-# check). When off, the server stays on the version currently installed:
-# _resolve_pip_spec pins the channel's dist to that version and the periodic
-# check is skipped. Governs the ha-mcp server package only — component updates
-# still come through HACS. An explicit OPT_PIP_SPEC override wins over both.
+# unpinned and auto-updates (force-install on reload/restart + a reload when the
+# periodic check sees a newer build). When off, the server stays on the version
+# currently installed: _resolve_pip_spec pins the channel's dist to that version
+# — but the periodic PyPI check KEEPS running so the update entity still shows
+# newer builds; its Install button is the manual path (issue #1760). Governs the
+# ha-mcp server package only — component updates still come through HACS. An
+# explicit OPT_PIP_SPEC override wins over both and skips the check entirely.
 OPT_AUTO_UPDATE = "auto_update"
 DEFAULT_AUTO_UPDATE = True
 OPT_SERVER_PORT = "server_port"
@@ -262,6 +283,14 @@
 # pre-release test channel) force an actual reinstall on the next start instead
 # of hitting the requirements manager's is-installed shortcut.
 DATA_LAST_PIP_SPEC = "last_pip_spec"
+# One-shot marker set by the update entity's Install button (issue #1760):
+# with auto-update off, EmbeddedServerManager._resolve_pip_spec pins the
+# channel to the CURRENTLY installed version, so a bare reload would just
+# reinstall the same build. This pins the next install to a specific version
+# regardless of auto_update; embedded_server clears it when it CONSUMES it
+# (before the install attempt) — one marker buys exactly one attempt, so a
+# failing pinned version can never re-pin later reloads (review finding).
+DATA_PENDING_INSTALL_VERSION = "pending_install_version"
 
 # hass.data[DOMAIN] sub-keys for the server runtime. Distinct from the tools
 # entry's sub-keys ("caller_token" / "allowed_paths") so both entry types can
@@ -273,6 +302,17 @@
 # on a genuine options change — the background bring-up persists ids/token/pip
 # spec to entry.data, and those writes must not trigger a self-reload.
 DATA_LAST_OPTIONS = "last_options"
+# The ServerVersionCoordinator instance backing the `update` platform entity
+# (issue #1760) — stored so the platform's async_setup_entry can retrieve it.
+DATA_UPDATE_COORDINATOR = "update_coordinator"
+# Set by async_maybe_auto_update right before it reloads the entry for an
+# automatic update ({"old": <version>}): the "server updated" notification must
+# only fire once the reloaded entry's bring-up actually installed and started
+# the new build — the reload call returns as soon as entry SETUP finishes,
+# while the pip install still runs in the background and can fail (review
+# finding on #1760). Bring-up pops it: notification on success, silent drop on
+# failure (the package/start repair issues cover that path).
+DATA_PENDING_UPDATE_NOTIFY = "pending_update_notify"
 
 # Webhook auth modes (mirrors the webhook-proxy add-on's default posture).
 WEBHOOK_AUTH_NONE = "none"  # secret webhook URL is the shared secret (default)
@@ -309,6 +349,14 @@
 # namespace (mirrors the webhook-proxy add-on's /api/mcp_proxy/oauth base).
 OAUTH_BASE = "/api/ha_mcp_tools/oauth"
 
+# HACS "add repository" deep link for the custom component. Shared learn_more_url
+# for every repair issue that ends with "install/reinstall the component via
+# HACS" (the component-outdated issue and the legacy-HACS-source issue below).
+HACS_COMPONENT_URL = (
+    "https://my.home-assistant.io/redirect/hacs_repository/"
+    "?owner=homeassistant-ai&repository=ha-mcp-integration&category=integration"
+)
+
 # Repair-issue ids surfaced when server bring-up fails.
 ISSUE_PACKAGE_FAILED = "server_package_install_failed"
 ISSUE_START_FAILED = "server_start_failed"
@@ -318,3 +366,11 @@
 # the server expects; this points the user at the HACS component update
 # (non-blocking).
 ISSUE_COMPONENT_OUTDATED = "component_outdated"
+# Repair issue surfaced when HACS is tracking the MAIN ha-mcp server repo for
+# this component (the pre-mirror install path — issue #1760). That install
+# keeps working (HACS downloads the repo snapshot at the release tag, which
+# contains the component), but HACS shows the SERVER's version numbers and
+# release notes, not the component's own; HACS has no repository-migration
+# mechanism, so this only self-resolves if the user re-adds the dedicated
+# mirror (homeassistant-ai/ha-mcp-integration).
+ISSUE_LEGACY_HACS_SOURCE = "legacy_hacs_source"

custom_components/ha_mcp_tools/coordinator.py

@@ -0,0 +1,112 @@
+"""Poll the server package's installed vs. latest PyPI version (issue #1760).
+
+Backs the ``update`` platform entity (:mod:`update`) and the automatic-update
+decision (:func:`embedded_setup.async_maybe_auto_update`). Runs on
+:data:`UPDATE_CHECK_INTERVAL` regardless of the ``auto_update`` option — unlike
+the check this replaces, visibility must not depend on auto-update being on
+(issue #1760: with auto-update off, users previously got zero signal that a
+server update existed). Only the resulting *reload* is gated on ``auto_update``,
+in :mod:`embedded_setup`.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+from aiohttp import ClientError
+from homeassistant.helpers.aiohttp_client import async_get_clientsession
+from homeassistant.helpers.update_coordinator import DataUpdateCoordinator
+
+from .const import (
+    DEFAULT_CHANNEL,
+    DEFAULT_PIP_SPEC,
+    DOMAIN,
+    OPT_CHANNEL,
+    OPT_PIP_SPEC,
+    PYPI_JSON_URL,
+    UPDATE_CHECK_INTERVAL,
+    dist_for_channel,
+)
+from .embedded_server import _installed_dist_version
+
+if TYPE_CHECKING:
+    from homeassistant.config_entries import ConfigEntry
+    from homeassistant.core import HomeAssistant
+
+_LOGGER = logging.getLogger(__name__)
+
+# Per-request timeout for the PyPI version-check fetch - short so a slow or
+# wedged PyPI never ties up the coordinator; a miss just retries next interval
+# (moved here from the old embedded_setup.async_check_for_update).
+_PYPI_TIMEOUT_SECONDS = 30
+
+
+@dataclass(frozen=True)
+class ServerVersionInfo:
+    """Installed vs. latest server-package version for one config entry."""
+
+    installed: str | None
+    latest: str | None
+    dist: str
+
+
+class ServerVersionCoordinator(DataUpdateCoordinator[ServerVersionInfo]):
+    """Poll the installed + PyPI-latest version of the in-process server package.
+
+    Deliberately NOT scoped to the ``auto_update`` option: the update entity
+    must stay populated and the periodic check must keep running even when the
+    user has automatic updates turned off - that visibility is the point of
+    issue #1760. ``embedded_entry`` schedules this coordinator's listener to
+    decide whether to actually reload.
+    """
+
+    def __init__(self, hass: HomeAssistant, entry: ConfigEntry) -> None:
+        """Bind to the config entry and schedule on UPDATE_CHECK_INTERVAL."""
+        self._entry = entry
+        super().__init__(
+            hass,
+            _LOGGER,
+            config_entry=entry,
+            name=f"{DOMAIN} server version",
+            update_interval=UPDATE_CHECK_INTERVAL,
+        )
+
+    async def _async_update_data(self) -> ServerVersionInfo:
+        """Return the installed + latest version for the configured channel.
+
+        Never raises ``UpdateFailed`` for an expected PyPI transient - the
+        entity must stay available (showing the installed version) even when
+        PyPI is unreachable; :meth:`_async_fetch_latest`'s own narrow except
+        clause is the only one expected to fire in normal operation.
+        """
+        options = self._entry.options
+        channel = str(options.get(OPT_CHANNEL) or DEFAULT_CHANNEL)
+        dist = dist_for_channel(channel)
+        installed = await self.hass.async_add_executor_job(
+            _installed_dist_version, dist
+        )
+
+        override = str(options.get(OPT_PIP_SPEC) or "").strip()
+        if override and override != DEFAULT_PIP_SPEC:
+            # An explicit pip-spec override (a version pin, a tarball URL)
+            # makes a PyPI-latest comparison meaningless - skip the fetch.
+            return ServerVersionInfo(installed=installed, latest=None, dist=dist)
+
+        latest = await self._async_fetch_latest(dist)
+        return ServerVersionInfo(installed=installed, latest=latest, dist=dist)
+
+    async def _async_fetch_latest(self, dist: str) -> str | None:
+        """Return the newest PyPI version for ``dist``, or None on any failure."""
+        try:
+            session = async_get_clientsession(self.hass)
+            async with asyncio.timeout(_PYPI_TIMEOUT_SECONDS):
+                async with session.get(PYPI_JSON_URL.format(dist=dist)) as resp:
+                    resp.raise_for_status()
+                    payload = await resp.json()
+            return str(payload["info"]["version"])
+        except (ClientError, TimeoutError, KeyError, ValueError) as err:
+            _LOGGER.debug("HA-MCP server version check failed for %s: %s", dist, err)
+            return None