Skip to content

Commit f7621d4

Browse files
test(e2e): drop redundant lifecycle roundtrips, keep only Matter Server (#1414) (#1419)
Closes #1414. The HAOS addon-lifecycle tier had four near-identical ``test_<addon>_start_stop_restart_roundtrip`` tests (Node-RED, ESPHome, Matter Server, AppDaemon). All four used the same ``_addon_action`` helper hitting the same MCP wire path — ``ha_call_service(hassio.addon_{start,stop,restart}, data={"addon": slug})`` — so they were four instances of one test parametrised by addon. From a code-coverage standpoint one is sufficient to guard the wire path. Node-RED's roundtrip in particular produced a recurring Supervisor 500 ``Server got itself in trouble`` flake on the first ``addon_stop`` call — four reproductions in <90 min across unrelated PRs, with reruns of the same commit consistently passing. Node-RED's container teardown (full Node.js runtime) is slow enough that Supervisor's addon_stop handler races to a 500 before the response body is built. Dropping three of the four roundtrips: * Removes the recurring Node-RED flake at source * Saves ~37-50s per HAOS lane (Node-RED ~25s, AppDaemon ~14s, ESPHome <13s) — based on pytest's slowest-10 across 15 recent runs * Preserves full addon-lifecycle wire coverage via the Matter Server roundtrip, which has never been observed flaking and runs <13s Matter Server kept because it's the lightest addon to cycle in the bake; AppDaemon (~14s, sometimes slowest-10) dropped along with the others. Addon-specific surface coverage is unchanged for every addon: ``ha_get_addon`` info/options, ``ha_manage_addon`` options writes, ``ha_get_logs`` Supervisor logs, and Node-RED's proxy/ingress tests (``test_manage_addon_modes.py``) all continue to run against the four running addons. Co-authored-by: kingpanther13 <kingpanther13@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e2067e4 commit f7621d4

1 file changed

Lines changed: 16 additions & 99 deletions

File tree

tests/src/e2e/haos_only/test_addon_lifecycle.py

Lines changed: 16 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,17 @@
77
Two coverage modes:
88
99
1. **Running addons** (Node-RED, ESPHome Device Builder, Matter Server,
10-
AppDaemon — all ``start=True`` in the bake): full lifecycle round-trip
11-
plus options-get / options-set persistence and a log-shape check.
12-
Each test leaves the addon in ``started`` state via an explicit
13-
cleanup call so later tests in the same session find it running.
10+
AppDaemon — all ``start=True`` in the bake): options-get / options-set
11+
persistence and a log-shape check. Each test leaves the addon in
12+
``started`` state via an explicit cleanup call so later tests in the
13+
same session find it running.
14+
15+
The full ``hassio.addon_stop`` → ``addon_start`` → ``addon_restart``
16+
lifecycle round-trip is exercised by **Matter Server only**. All four
17+
roundtrip variants used the same ``_addon_action`` helper hitting the
18+
same MCP wire path, so one is sufficient signal; Matter Server is the
19+
lightest addon to cycle (consistently <13s) and Node-RED's roundtrip
20+
produced a recurring Supervisor 500 flake — see #1414.
1421
1522
2. **Stopped addons** (Mosquitto, MQTT IO — stay ``start=False`` because
1623
their schemas require config they don't have in the bake): exercise
@@ -87,9 +94,7 @@ async def _resolve_slug(mcp_client: Any, display_name: str) -> str:
8794
for entry in payload.get("addons", []):
8895
if entry.get("name") == display_name:
8996
slug = entry.get("slug")
90-
assert slug, (
91-
f"Addon {display_name!r} listed but has no slug field: {entry}"
92-
)
97+
assert slug, f"Addon {display_name!r} listed but has no slug field: {entry}"
9398
return str(slug)
9499

95100
# Filter ``None`` names so a future Supervisor addition with a
@@ -116,9 +121,7 @@ async def _get_addon_detail(mcp_client: Any, slug: str) -> dict[str, Any]:
116121
return detail
117122

118123

119-
async def _addon_action(
120-
mcp_client: Any, slug: str, action: str
121-
) -> dict[str, Any]:
124+
async def _addon_action(mcp_client: Any, slug: str, action: str) -> dict[str, Any]:
122125
"""Invoke ``hassio.addon_{action}`` via ``ha_call_service``.
123126
124127
Returns the parsed result (success or failure). Caller decides how
@@ -209,36 +212,6 @@ async def _ensure_started(mcp_client: Any, slug: str) -> None:
209212
# ---------------------------------------------------------------------------
210213

211214

212-
async def test_nodered_start_stop_restart_roundtrip(mcp_client: Any) -> None:
213-
"""Stop → start → restart cycles Node-RED via real Supervisor.
214-
215-
Each transition is verified by re-reading ``ha_get_addon(slug=...)``
216-
and asserting the state moved as expected. Final assertion plus the
217-
cleanup hook leave the addon ``started`` for downstream tests.
218-
"""
219-
slug = await _resolve_slug(mcp_client, NODERED_NAME)
220-
try:
221-
stop_result = await _addon_action(mcp_client, slug, "stop")
222-
assert stop_result.get("success"), (
223-
f"hassio.addon_stop({slug}) failed: {stop_result}"
224-
)
225-
await _wait_for_state(mcp_client, slug, STOPPED_STATES)
226-
227-
start_result = await _addon_action(mcp_client, slug, "start")
228-
assert start_result.get("success"), (
229-
f"hassio.addon_start({slug}) failed: {start_result}"
230-
)
231-
await _wait_for_state(mcp_client, slug, "started")
232-
233-
restart_result = await _addon_action(mcp_client, slug, "restart")
234-
assert restart_result.get("success"), (
235-
f"hassio.addon_restart({slug}) failed: {restart_result}"
236-
)
237-
await _wait_for_state(mcp_client, slug, "started")
238-
finally:
239-
await _ensure_started(mcp_client, slug)
240-
241-
242215
async def test_nodered_options_get_returns_dict(mcp_client: Any) -> None:
243216
"""`ha_get_addon(slug=...)` exposes the addon's options as a dict."""
244217
slug = await _resolve_slug(mcp_client, NODERED_NAME)
@@ -295,9 +268,7 @@ async def test_nodered_options_set_persists(mcp_client: Any) -> None:
295268
ok = write_payload.get("success") is True or (
296269
write_payload.get("status") == "pending_restart"
297270
)
298-
assert ok, (
299-
f"ha_manage_addon options probe write failed: {write_payload}"
300-
)
271+
assert ok, f"ha_manage_addon options probe write failed: {write_payload}"
301272

302273
detail_after = await _get_addon_detail(mcp_client, slug)
303274
options_after = detail_after.get("options") or {}
@@ -344,31 +315,6 @@ async def test_nodered_logs_fetch_shape(mcp_client: Any) -> None:
344315
# ---------------------------------------------------------------------------
345316

346317

347-
async def test_esphome_start_stop_restart_roundtrip(mcp_client: Any) -> None:
348-
"""Stop → start → restart cycles ESPHome via real Supervisor."""
349-
slug = await _resolve_slug(mcp_client, ESPHOME_NAME)
350-
try:
351-
stop_result = await _addon_action(mcp_client, slug, "stop")
352-
assert stop_result.get("success"), (
353-
f"hassio.addon_stop({slug}) failed: {stop_result}"
354-
)
355-
await _wait_for_state(mcp_client, slug, STOPPED_STATES)
356-
357-
start_result = await _addon_action(mcp_client, slug, "start")
358-
assert start_result.get("success"), (
359-
f"hassio.addon_start({slug}) failed: {start_result}"
360-
)
361-
await _wait_for_state(mcp_client, slug, "started")
362-
363-
restart_result = await _addon_action(mcp_client, slug, "restart")
364-
assert restart_result.get("success"), (
365-
f"hassio.addon_restart({slug}) failed: {restart_result}"
366-
)
367-
await _wait_for_state(mcp_client, slug, "started")
368-
finally:
369-
await _ensure_started(mcp_client, slug)
370-
371-
372318
async def test_esphome_options_get_returns_dict(mcp_client: Any) -> None:
373319
"""`ha_get_addon(slug=...)` exposes ESPHome's options as a dict."""
374320
slug = await _resolve_slug(mcp_client, ESPHOME_NAME)
@@ -417,9 +363,7 @@ async def test_esphome_options_set_persists(mcp_client: Any) -> None:
417363
ok = write_payload.get("success") is True or (
418364
write_payload.get("status") == "pending_restart"
419365
)
420-
assert ok, (
421-
f"ha_manage_addon options probe write failed: {write_payload}"
422-
)
366+
assert ok, f"ha_manage_addon options probe write failed: {write_payload}"
423367

424368
detail_after = await _get_addon_detail(mcp_client, slug)
425369
options_after = detail_after.get("options") or {}
@@ -540,9 +484,7 @@ async def test_matter_server_options_set_persists(mcp_client: Any) -> None:
540484
ok = write_payload.get("success") is True or (
541485
write_payload.get("status") == "pending_restart"
542486
)
543-
assert ok, (
544-
f"ha_manage_addon options probe write failed: {write_payload}"
545-
)
487+
assert ok, f"ha_manage_addon options probe write failed: {write_payload}"
546488

547489
detail_after = await _get_addon_detail(mcp_client, slug)
548490
options_after = detail_after.get("options") or {}
@@ -582,31 +524,6 @@ async def test_matter_server_logs_fetch_shape(mcp_client: Any) -> None:
582524
# ---------------------------------------------------------------------------
583525

584526

585-
async def test_appdaemon_start_stop_restart_roundtrip(mcp_client: Any) -> None:
586-
"""Stop → start → restart cycles AppDaemon via real Supervisor."""
587-
slug = await _resolve_slug(mcp_client, APPDAEMON_NAME)
588-
try:
589-
stop_result = await _addon_action(mcp_client, slug, "stop")
590-
assert stop_result.get("success"), (
591-
f"hassio.addon_stop({slug}) failed: {stop_result}"
592-
)
593-
await _wait_for_state(mcp_client, slug, STOPPED_STATES)
594-
595-
start_result = await _addon_action(mcp_client, slug, "start")
596-
assert start_result.get("success"), (
597-
f"hassio.addon_start({slug}) failed: {start_result}"
598-
)
599-
await _wait_for_state(mcp_client, slug, "started")
600-
601-
restart_result = await _addon_action(mcp_client, slug, "restart")
602-
assert restart_result.get("success"), (
603-
f"hassio.addon_restart({slug}) failed: {restart_result}"
604-
)
605-
await _wait_for_state(mcp_client, slug, "started")
606-
finally:
607-
await _ensure_started(mcp_client, slug)
608-
609-
610527
async def test_appdaemon_webui_no_ingress_shape(mcp_client: Any) -> None:
611528
"""`ha_get_addon(slug=...)` reports ``webui`` set + ``ingress=false`` for AppDaemon.
612529

0 commit comments

Comments
 (0)