Skip to content

Commit d3295a8

Browse files
kingpanther13claude
kingpanther13
and
claude
authored
test: add an update-path e2e lane covering released-component x new-server installs (#1796)
* test: add update-path e2e reproducing component x new-server updates Boots a dedicated HA container with the ha_mcp_tools custom component and the ha-mcp server installed from PyPI stable, then drives the component's real in-process update mechanism (options-flow pip_spec override -> entry reload -> force install -> module purge -> worker restart) to a wheel built from the checkout with a sentinel version, and asserts the server comes back up reporting that sentinel and serving the full tool inventory. Two scenarios run per job: the component at the released `stable` git tag (proves a new server release won't break existing installs - the sequence issues #1783/#1785 regressed on) and the working-tree component (proves the PR's own update machinery still works, caught on the introducing PR). The module runs only in its dedicated CI lane (E2E_UPDATE_PATH=1 + -m update_path); the broad e2e lanes collect it and skip at zero cost. In the dedicated lane an unreachable Docker daemon FAILS the fixture instead of skipping - a fixture skip exits pytest 0 and would let the required gate go green while testing nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWhJZ38Z1af1AjDDTtoZbP * ci: run the update-path e2e as a dedicated required lane New single-arch jobs in pr.yml (folded into the required e2e-validation-gate) and e2e-tests.yml running only test_embedded_update_path.py with E2E_UPDATE_PATH=1; both scenarios (component@stable, component@working-tree) run serially in one job. fetch-depth 0 with a tag sanity step because the stable scenario checks the component out of the `stable` tag; 30m budget covers both bring-ups' PyPI installs; E2E_BACKEND left unset so the autouse session backend boots the cheap container variant instead of an unused embedded preinstall. Docs-only PRs skip it like the other e2e lanes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWhJZ38Z1af1AjDDTtoZbP --------- Co-authored-by: kingpanther13 <kingpanther13@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 2fd1621 commit d3295a8

4 files changed

Lines changed: 883 additions & 5 deletions

File tree

‎.github/workflows/e2e-tests.yml‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,3 +193,105 @@ jobs:
193193
E2E_BACKEND: "embedded"
194194
HAMCP_ENV_FILE: "tests/.env.test"
195195
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
196+
197+
# Update-path e2e lane (#1783/#1785): mirrors pr.yml's e2e-validation-update-path
198+
# on master push. Each run parametrizes over the component source — the released
199+
# ``stable`` git tag (proves a new server release won't break existing installs)
200+
# AND this checkout's working tree (proves the PR's own update machinery still
201+
# works) — with the ha-mcp server installed from PyPI stable in both. The test
202+
# then drives the in-process update that installs THIS commit's freshly built
203+
# wheel over the running server (the real reload/purge/restart path) and asserts
204+
# the server survives. A single ubuntu-latest lane runs both scenarios serially
205+
# (no -n), not the parallel suite. fetch-depth 0 is REQUIRED — the default
206+
# shallow checkout omits the ``stable`` tag the stable scenario checks out from.
207+
e2e-tests-update-path:
208+
name: E2E Tests (Update Path)
209+
runs-on: ubuntu-latest
210+
# 30m: each of the two scenarios (component@stable and component@working-tree)
211+
# does a first bring-up that installs ha-mcp plus the full fastmcp dependency
212+
# tree from PyPI stable before the in-process update swaps in the PR wheel. The
213+
# two containers run sequentially in this one job (observed ~50s per scenario),
214+
# so 30m still fits comfortably.
215+
timeout-minutes: 30
216+
217+
steps:
218+
- uses: actions/checkout@v7
219+
with:
220+
submodules: true
221+
# fetch-depth 0: the update-path scenario checks the custom component
222+
# out at the released ``stable`` git tag, which the default shallow
223+
# checkout does not fetch.
224+
fetch-depth: 0
225+
226+
- name: Verify stable tag is present
227+
run: |
228+
if ! git rev-parse --verify refs/tags/stable >/dev/null 2>&1; then
229+
echo "::error::The 'stable' git tag is missing — the update-path lane checks the released component out from it. Ensure checkout uses fetch-depth: 0."
230+
exit 1
231+
fi
232+
233+
- name: Set up Docker Buildx
234+
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
235+
with:
236+
cache-binary: true
237+
238+
- name: Install uv
239+
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
240+
with:
241+
version: "latest"
242+
243+
- name: Set up Python
244+
run: uv python install ${{ env.PYTHON_VERSION }}
245+
246+
- name: Install dependencies
247+
run: uv sync --all-extras --dev
248+
249+
- name: Cache HA Docker image
250+
id: cache-ha-image
251+
uses: actions/cache@v6
252+
with:
253+
path: /tmp/ha-image.tar
254+
key: ha-image-${{ env.HA_IMAGE_GHCR }}-${{ runner.arch }}
255+
256+
- name: Load cached HA image
257+
if: steps.cache-ha-image.outputs.cache-hit == 'true'
258+
run: docker load -i /tmp/ha-image.tar
259+
260+
- name: Pull HA image (GHCR → Docker Hub fallback)
261+
if: steps.cache-ha-image.outputs.cache-hit != 'true'
262+
run: |
263+
HA_VERSION="${HA_IMAGE_GHCR##*:}"
264+
HA_IMAGE_DOCKERHUB="homeassistant/home-assistant:${HA_VERSION}"
265+
for registry in "$HA_IMAGE_GHCR" "$HA_IMAGE_DOCKERHUB"; do
266+
echo "Trying $registry..."
267+
if docker pull "$registry"; then
268+
if [ "$registry" != "$HA_IMAGE_GHCR" ]; then
269+
docker tag "$registry" "$HA_IMAGE_GHCR"
270+
fi
271+
docker save "$HA_IMAGE_GHCR" -o /tmp/ha-image.tar
272+
echo "Pulled and cached from $registry"
273+
exit 0
274+
fi
275+
echo "Failed to pull from $registry, trying next..."
276+
sleep 15
277+
done
278+
echo "All registries failed" && exit 1
279+
280+
- name: Run update-path e2e test
281+
# E2E_BACKEND deliberately UNSET (container backend): the test drives its
282+
# own dedicated container end to end and never touches the session
283+
# backend, but conftest's autouse session fixture boots one regardless —
284+
# the default container backend boots it cheaply, while
285+
# E2E_BACKEND=embedded would add a wheel build + in-container preinstall
286+
# of the whole fastmcp tree that nothing in this lane uses.
287+
run: |
288+
echo "🚀 Running the update-path e2e lane (released component + PyPI server → PR wheel)..."
289+
uv run pytest tests/src/e2e/workflows/embedded/test_embedded_update_path.py \
290+
-m update_path \
291+
--tb=short \
292+
-v
293+
echo "✅ Update-path e2e lane passed"
294+
env:
295+
E2E_UPDATE_PATH: "1"
296+
HAMCP_ENV_FILE: "tests/.env.test"
297+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/pr.yml‎

Lines changed: 117 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ jobs:
8383
# variants: it always runs, passes when the suite was skipped (docs/website)
8484
# or every variant succeeded, and fails if any variant failed.
8585
name: E2E Validation Gate
86-
needs: [changes, e2e-validation, e2e-validation-embedded]
86+
needs: [changes, e2e-validation, e2e-validation-embedded, e2e-validation-update-path]
8787
if: always()
8888
runs-on: ubuntu-latest
8989
timeout-minutes: 2
@@ -96,12 +96,15 @@ jobs:
9696
fi
9797
container_result="${{ needs.e2e-validation.result }}"
9898
embedded_result="${{ needs.e2e-validation-embedded.result }}"
99+
update_path_result="${{ needs.e2e-validation-update-path.result }}"
99100
echo "e2e-validation result: $container_result"
100101
echo "e2e-validation-embedded result: $embedded_result"
101-
# Both the container-backend matrix AND the in-process MCP server
102-
# lane must pass. Either failing wedges the required gate.
103-
if [ "$container_result" != "success" ] || [ "$embedded_result" != "success" ]; then
104-
echo "::error::E2E Validation did not succeed (container=$container_result, embedded=$embedded_result)."
102+
echo "e2e-validation-update-path result: $update_path_result"
103+
# The container-backend matrix, the in-process MCP server lane, AND
104+
# the update-path lane must all pass. Any one failing wedges the
105+
# required gate.
106+
if [ "$container_result" != "success" ] || [ "$embedded_result" != "success" ] || [ "$update_path_result" != "success" ]; then
107+
echo "::error::E2E Validation did not succeed (container=$container_result, embedded=$embedded_result, update-path=$update_path_result)."
105108
exit 1
106109
fi
107110
@@ -513,6 +516,115 @@ jobs:
513516
HAMCP_ENV_FILE: "tests/.env.test"
514517
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
515518

519+
# Update-path e2e lane (#1783/#1785): reproduces the in-place server update
520+
# that broke on real installs. Each run parametrizes over the component source
521+
# — the released ``stable`` git tag (proves a new server release won't break
522+
# existing installs) AND this PR's working tree (proves the PR's own update
523+
# machinery still works) — with the ha-mcp server installed from PyPI stable in
524+
# both. The test then drives the in-process update that installs THIS PR's
525+
# freshly built wheel over the running server — exercising the real
526+
# reload/purge/restart path — and asserts the server survives. A single
527+
# ubuntu-latest lane runs both scenarios serially (no -n), not the parallel
528+
# suite. fetch-depth 0 is REQUIRED — the default shallow checkout omits the
529+
# ``stable`` tag the stable scenario checks out from. Gated like the other e2e
530+
# lanes: skips on docs/website-only PRs (needs.changes) and folds into
531+
# e2e-validation-gate.
532+
e2e-validation-update-path:
533+
name: E2E Validation (update path)
534+
needs: changes
535+
# A skipped job reports Success, so a docs/website-only PR doesn't wedge the
536+
# gate waiting on this lane (mirrors e2e-validation).
537+
if: needs.changes.outputs.run == 'true'
538+
runs-on: ubuntu-latest
539+
# 30m: each of the two scenarios (component@stable and component@working-tree)
540+
# does a first bring-up that installs ha-mcp plus the full fastmcp dependency
541+
# tree from PyPI stable before the in-process update swaps in the PR wheel. The
542+
# two containers run sequentially in this one job (observed ~50s per scenario),
543+
# so 30m still fits comfortably.
544+
timeout-minutes: 30
545+
546+
steps:
547+
- uses: actions/checkout@v7
548+
with:
549+
submodules: true
550+
# fetch-depth 0: the update-path scenario checks the custom component
551+
# out at the released ``stable`` git tag, which the default shallow
552+
# checkout does not fetch.
553+
fetch-depth: 0
554+
555+
- name: Verify stable tag is present
556+
run: |
557+
if ! git rev-parse --verify refs/tags/stable >/dev/null 2>&1; then
558+
echo "::error::The 'stable' git tag is missing — the update-path lane checks the released component out from it. Ensure checkout uses fetch-depth: 0."
559+
exit 1
560+
fi
561+
562+
- name: Set up Docker Buildx
563+
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
564+
with:
565+
cache-binary: true
566+
567+
- name: Install uv
568+
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
569+
with:
570+
version: "latest"
571+
572+
- name: Set up Python
573+
run: uv python install ${{ env.PYTHON_VERSION }}
574+
575+
- name: Install dependencies
576+
run: uv sync --all-extras --dev
577+
578+
- name: Cache HA Docker image
579+
id: cache-ha-image
580+
uses: actions/cache@v6
581+
with:
582+
path: /tmp/ha-image.tar
583+
key: ha-image-${{ env.HA_IMAGE_GHCR }}-${{ runner.arch }}
584+
585+
- name: Load cached HA image
586+
if: steps.cache-ha-image.outputs.cache-hit == 'true'
587+
run: docker load -i /tmp/ha-image.tar
588+
589+
- name: Pull HA image (GHCR → Docker Hub fallback)
590+
if: steps.cache-ha-image.outputs.cache-hit != 'true'
591+
run: |
592+
HA_VERSION="${HA_IMAGE_GHCR##*:}"
593+
HA_IMAGE_DOCKERHUB="homeassistant/home-assistant:${HA_VERSION}"
594+
for registry in "$HA_IMAGE_GHCR" "$HA_IMAGE_DOCKERHUB"; do
595+
echo "Trying $registry..."
596+
if docker pull "$registry"; then
597+
if [ "$registry" != "$HA_IMAGE_GHCR" ]; then
598+
docker tag "$registry" "$HA_IMAGE_GHCR"
599+
fi
600+
docker save "$HA_IMAGE_GHCR" -o /tmp/ha-image.tar
601+
echo "Pulled and cached from $registry"
602+
exit 0
603+
fi
604+
echo "Failed to pull from $registry, trying next..."
605+
sleep 15
606+
done
607+
echo "All registries failed" && exit 1
608+
609+
- name: Run update-path e2e test
610+
# E2E_BACKEND deliberately UNSET (container backend): the test drives its
611+
# own dedicated container end to end and never touches the session
612+
# backend, but conftest's autouse session fixture boots one regardless —
613+
# the default container backend boots it cheaply, while
614+
# E2E_BACKEND=embedded would add a wheel build + in-container preinstall
615+
# of the whole fastmcp tree that nothing in this lane uses.
616+
run: |
617+
echo "🚀 Running the update-path e2e lane (released component + PyPI server → PR wheel)..."
618+
uv run pytest tests/src/e2e/workflows/embedded/test_embedded_update_path.py \
619+
-m update_path \
620+
--tb=short \
621+
-v
622+
echo "✅ Update-path e2e lane passed"
623+
env:
624+
E2E_UPDATE_PATH: "1"
625+
HAMCP_ENV_FILE: "tests/.env.test"
626+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
627+
516628
# Docker and Add-on validation
517629
docker-validation:
518630
name: Docker & Add-on Validation

‎tests/pytest.ini‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ markers =
5050
zone: zone management tests
5151
group: entity group management tests
5252
hacs: HACS (Home Assistant Community Store) tests
53+
update_path: component (stable tag + working tree) x new-server in-process update-path e2e (dedicated lane; runs only with E2E_UPDATE_PATH=1; #1783/#1785)
5354
core: core tool tests (state, service, bulk control, history, templates)
5455
config: configuration management tests (helpers, labels)
5556
filesystem: filesystem access tools tests

0 commit comments

Comments
 (0)