Skip to content

Commit 67ba242

Browse files
committed
test(e2e): assert backend dispatch matches workflow env vars on every lane
Adds a single smoke test at tests/src/e2e/basic/test_backend_dispatch_smoke.py that runs on all three e2e lanes (testcontainer, HAOS external, HAOS inaddon) and asserts ha_container_with_fresh_config["backend"] matches what the HAOS_TEST_IMAGE_PATH and HAOS_TEST_MODE env vars imply. Lane → expected backend: e2e-tests.yml (testcontainer): container haos-e2e-tests.yml (external): haos haos-e2e-inaddon-tests.yml: haos_inaddon Silent-failure modes this catches (would have left previous CI green): - HAOS_TEST_IMAGE_PATH set but conftest falls through to testcontainer (the lane is unknowingly testing the wrong HA instance) - HAOS_TEST_MODE=inaddon ignored, inaddon lane silently runs external dispatch — the addon integration is never actually exercised - haos_inaddon backend reached but addon_mcp_url missing, leaving mcp_client fixtures pointing at the wrong endpoint Placed under basic/ (NOT haos_only/) on purpose: the auto-applied haos_only marker would skip the test when is_haos_backend_selected() returns False, which is exactly the silent-failure case we want to catch. basic/ has no auto-applied markers so the test runs everywhere.
1 parent 401b7b4 commit 67ba242

1 file changed

Lines changed: 87 additions & 0 deletions

File tree

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
"""Verifies the conftest backend dispatch picks the backend CI asked for.
2+
3+
The three e2e CI lanes set env vars that ``conftest.ha_container_with_fresh_config``
4+
reads to choose a backend:
5+
6+
| Lane | HAOS_TEST_IMAGE_PATH | HAOS_TEST_MODE | expected backend |
7+
| ----------------------------- | -------------------- | -------------- | ---------------- |
8+
| e2e-tests.yml (testcontainer) | unset | unset | ``container`` |
9+
| haos-e2e-tests.yml (external) | set | unset | ``haos`` |
10+
| haos-e2e-inaddon-tests.yml | set | ``inaddon`` | ``haos_inaddon`` |
11+
12+
Without an explicit assertion, the dispatch has silent-failure modes that
13+
all leave CI green while running tests against the wrong backend:
14+
15+
1. ``HAOS_TEST_IMAGE_PATH`` set but ``is_haos_backend_selected()`` returns
16+
False (env-var name drift, import-time bug) — both HAOS lanes silently
17+
fall through to the testcontainer path.
18+
2. ``HAOS_TEST_MODE=inaddon`` set but ``is_haos_inaddon_mode()`` reads a
19+
different name — inaddon lane silently runs the external dispatch,
20+
so ``mcp_client`` talks to the in-process FastMCP server instead of
21+
the addon's HTTP endpoint. The whole inaddon integration is untested.
22+
3. Inaddon dispatch reached but ``addon_mcp_url`` never populated —
23+
downstream fixtures route to the wrong endpoint and surface as
24+
confusing errors later.
25+
26+
This file is placed under ``basic/`` (NOT ``haos_only/``) on purpose:
27+
the auto-applied ``haos_only`` marker in ``conftest.pytest_collection_modifyitems``
28+
would skip the test whenever ``is_haos_backend_selected()`` returns False,
29+
which is exactly the silent-failure case (1) above. We want the test to
30+
RUN on every lane and FAIL when the backend doesn't match the env.
31+
"""
32+
33+
from __future__ import annotations
34+
35+
import os
36+
from typing import Any
37+
38+
39+
def test_backend_dispatch_matches_workflow_env(
40+
ha_container_with_fresh_config: dict[str, Any],
41+
) -> None:
42+
"""Conftest dispatch must pick the backend the workflow env implies.
43+
44+
Runs unconditionally on every lane — assertion branches off the env
45+
vars to mirror conftest's own dispatch logic. Mismatch means the
46+
dispatch silently picked a different backend than CI asked for.
47+
"""
48+
image_path = os.environ.get("HAOS_TEST_IMAGE_PATH")
49+
mode = os.environ.get("HAOS_TEST_MODE", "")
50+
backend = ha_container_with_fresh_config["backend"]
51+
52+
if image_path and mode == "inaddon":
53+
# haos-e2e-inaddon-tests.yml lane
54+
assert backend == "haos_inaddon", (
55+
f"Workflow set HAOS_TEST_IMAGE_PATH + HAOS_TEST_MODE=inaddon "
56+
f"but dispatch picked backend={backend!r}. The inaddon "
57+
f"integration is NOT being exercised by this run."
58+
)
59+
addon_mcp_url = ha_container_with_fresh_config.get("addon_mcp_url")
60+
assert addon_mcp_url and addon_mcp_url.startswith("http"), (
61+
f"haos_inaddon backend reported but addon_mcp_url is "
62+
f"{addon_mcp_url!r}. mcp_client fixtures will route to the "
63+
f"wrong endpoint."
64+
)
65+
assert ha_container_with_fresh_config["container"] is None
66+
elif image_path:
67+
# haos-e2e-tests.yml (external) lane
68+
assert backend == "haos", (
69+
f"Workflow set HAOS_TEST_IMAGE_PATH but dispatch picked "
70+
f"backend={backend!r}. The lane silently fell through to "
71+
f"the testcontainer path; tests are running against the "
72+
f"wrong HA instance."
73+
)
74+
# External HAOS sets the testcontainer keys to None.
75+
assert ha_container_with_fresh_config["container"] is None
76+
assert ha_container_with_fresh_config["port"] is None
77+
assert ha_container_with_fresh_config["config_path"] is None
78+
# addon_mcp_url is the inaddon-only routing key.
79+
assert ha_container_with_fresh_config["addon_mcp_url"] is None
80+
else:
81+
# e2e-tests.yml (testcontainer) lane
82+
assert backend == "container", (
83+
f"No HAOS env vars set, expected testcontainer backend, "
84+
f"got backend={backend!r}."
85+
)
86+
assert ha_container_with_fresh_config["container"] is not None
87+
assert ha_container_with_fresh_config["port"] is not None

0 commit comments

Comments
 (0)