Commit f7d1b5a
test(internal): JSDOM behaviour harness + auto-discovery parse coverage for every rendered <script> (#1425)
* test(internal): JSDOM behaviour harness + auto-discovery parse coverage for every rendered <script>
Closes #1422.
Adds a JSDOM harness (tests/js/harness.mjs + tests/src/unit/_js_harness.py)
that drives real rendered <script> bodies through stubbed fetch /
BroadcastChannel / virtual timers / DOM and reports observed side
effects. A discovery walker auto-picks-up every <script> surface in
the repo (src/ha_mcp/settings_ui.py, src/ha_mcp/auth/consent_form.py,
every site/src/**/*.astro) so parse coverage extends as new UI surfaces
ship — no registration needed.
Behavioural coverage landed for the surfaces named in #1422:
* settings_ui — restartInProgress concurrency guard, 4xx-suppress-reload
branch, 5xx fall-through, instance_id-flip probe, BroadcastChannel
restart-required + restart-initiated listeners, saveFeatureFlag
JSON-parse fallback.
* setup.astro — state-machine progression (local / network / remote),
plus a parametrised per-client smoke that drives the wizard to
config generation for every id in the real clientsData array.
* tools.astro — search/filter pipeline + design-mode toggle (TypeScript;
esbuild strips types in the harness before eval).
* Layout.astro — copy-button idempotency across re-init.
* consent_form — submit handler disable + spinner state.
The legacy TestRenderedHTMLJsSyntax in test_settings_ui.py is removed —
the auto-discovery parse test in test_rendered_scripts_parse.py
subsumes it (and extends to the four other surfaces it never covered).
CI: unit-tests job in pr.yml installs nodejs + jsdom + esbuild via
apt-get / npm ci. Local devs without tests/js/node_modules/ get clean
skips, matching the original parse guard's behaviour.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): skip Astro frontmatter in script extraction; drain microtasks before clock advance
CI surfaced two harness bugs the local smoke-tests didn't catch:
1. `extract_script_body` and the discovery walker greedily matched the
first `<script>` substring in the source, which in setup.astro is
actually a frontmatter comment: `// below in the <script> block keyed
off the entry's id.` That made the "script body" start mid-frontmatter
and the extracted text wasn't valid JS — esbuild and JSDOM both
rejected it with "Unexpected identifier 'keyed'".
Fix: strip the `--- ... ---` Astro frontmatter block before searching
for `<script>` tags. Plain .py and .html sources have no frontmatter
and pass through unchanged.
2. `clock.advance(settleMs)` returned immediately when no timers were
yet scheduled, but the script under test often awaits a chain of
stubbed-fetch promises BEFORE hitting its first `setTimeout`. With
only one microtask drain between eval and advance, those promises
hadn't resolved yet, so no timers existed, advance was a no-op, and
the script stayed suspended — `restartAddon`'s POST to
/api/settings/restart never fired and the `alert(msg)` in the 4xx
branch never ran.
Fix: drain microtasks aggressively at the start of advance() so
pending promises get to schedule their timers, and drain again when
the timer queue temporarily empties (a promise resolution may queue
new timers).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(internal): expand initial DOM fixtures to cover every top-level addEventListener target
CI surfaced this via the test_5xx test (the only one whose assertion
included the harness errors list): the settings_ui script aborts during
init at `document.getElementById('backupRefresh').addEventListener(...)`
because the test DOM is missing the backup table / modal markup. With
init aborted, the invoke step never runs — `restartAddon` is never
called, POSTs never fire, `alert()` never runs, and all three restart-
flow tests silently fail.
The setup.astro tests had the same shape: `generateConfig` queries
`config-section` (distinct from `section-config`) to show/hide the
inner code block. Without it, the proxy click handler in the remote-
flow test threw and the `document.body.dataset.beforeProxy` assignment
never landed.
Fixes:
- settings_ui MIN_DOM now includes backupBulkDelete, backupDomain,
backupEntity, backupList, backupRefresh, backupState, featuresBody,
modalBackdrop / modalBody / modalClose / modalTitle. Set built from
`grep -h "document.getElementById" settings_ui.py` so future top-level
handlers will surface as the same pattern.
- setup.astro DOM now includes config-section alongside section-config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): run JSDOM eval at global scope; capture body attrs in dom snapshot
Two harness bugs the prior CI rounds didn't surface until init-stage
crashes were resolved:
1. Wrapping the rendered script in an `async () => { ... }()` IIFE
confined top-level `function` declarations to the IIFE scope.
`function restartAddon() {...}` never landed on `window`, so
`invoke: "window.restartAddon();"` threw `is not a function`. A
real browser hoists inline-script function decls to the global
window — match that by running prelude + script body at global
scope and keeping the IIFE for `invoke` alone (so awaits inside
invoke still work).
2. `document.body.innerHTML` returns body's children but not body's
own attrs, so tests that wrote `document.body.dataset.foo = 'bar'`
as a side-channel for in-page state had no way to assert on it —
`result.dom` came back without the attr. Serialise
`document.documentElement.outerHTML` instead so html/head/body
tags and their own attributes round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(internal): sequence /api/settings/info responses for 5xx restart probe
The test_5xx flow hits the info endpoint three times — loadTools init,
restartAddon's pre-POST baseline capture, and _probeAddonRestarted
after the POST. The old single-response fixture returned the SAME
instance_id every time, so the probe never saw the flip and looped
until timeout, leaving reloads=0.
Adds a `responses: [...]` shape to the harness fetch_map: each match
on a URL pattern advances a per-pattern counter; the last entry sticks
after exhaustion (matches "the addon came back online and stays
online"). Test now provides baseline → baseline → flipped so the
probe terminates with restarted=true and the reload fires.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(test): cache apt downloads and node_modules for the unit-tests job
The unit-tests "Install git and Node.js" step was 44 s — almost all of
it network download of the nodejs / npm .deb. The "Install JS test
dependencies" step is 1 s when node_modules is fresh but can grow as
deps change.
- Cache /var/cache/apt/archives keyed on a stable string (apt package
set rarely changes). Disable docker-clean and set Keep-Downloaded-
Packages so the cached .debs survive install for the next run. apt
install still runs (unpacks from local cache, ~3-5 s) but skips the
network leg.
- Cache tests/js/node_modules keyed on package-lock.json so dep bumps
invalidate cleanly. `npm ci` short-circuits when the tree matches.
Expected first-cold-cache run: unchanged (~45 s install). Cache hits:
~5 s for both steps combined.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(internal): address Gemini + pr-review-toolkit findings on JS test infrastructure
Outcome of going through all 60 findings from Gemini Code Assist +
pr-review-toolkit (code-reviewer, pr-test-analyzer, silent-failure-
hunter, comment-analyzer). 3 wrong (skipped: PATH-resolved node binary
mis-flagged as hardcoded; project-relative esbuild path mis-flagged
as hardcoded; theoretical FakeBroadcastChannel constructor-throw).
Remaining real items addressed:
Harness:
- vm.runInContext replaces window.eval / indirect eval to clear the
"no eval()" style-guide flag (Gemini #1, #2).
- Timer-callback and broadcast-listener throws now record into the
errors list instead of being silently swallowed (#30, #32).
- SAFETY_CAP exhaustion records a clear "runaway setInterval" error
instead of breaking silently (#4, #31).
- Non-navigation jsdomErrors route to errors (not console) so tests
asserting `not result.errors` catch them (#38).
- Transpile failure short-circuits init eval to avoid cascading
syntax errors from un-transpiled TS (#34).
- FakeBroadcastChannel.postMessage now delivers to peer same-name
channels in the same context per spec (#5).
- Time-faked surface documented accurately (Date.now / setTimeout /
setInterval only; new Date / performance.now still wall-time) (#46).
- New broadcastChannelUnavailable param simulates the
`typeof BroadcastChannel === 'undefined'` browsing context so the
production null-guard branch is exercised (#15).
- Dead comments and rot-prone duplications removed (#47, #49, #51,
#56, #57, #58, #59, #66).
extract_astro_vars.mjs:
- vm.runInContext replaces (0, eval) (Gemini #2).
- Multi-line `import { a, b } from 'x';` now stripped robustly (#7).
- Eval errors wrapped with the source path for actionable failures (#35).
_js_harness.py:
- Wrong test file name and workflow path in docstring fixed (#41, #42).
- _strip_astro_frontmatter raises ValueError when frontmatter opens but
never closes (#36).
- discover_script_surfaces raises when site/src/ is missing instead of
silently producing partial results (#37).
- extract_script_body accepts source_label for actionable errors (#40).
- Astro `<script lang="js">` is no longer mis-tagged as TypeScript (#9).
- Inert chr(92) Windows backslash replace removed (#14).
- Field docstrings on ScriptSurface trimmed to the one that earns its
keep (#52).
- _PY_RENDERERS registry refactor + accurate enumeration comment (#45).
test_settings_ui_js_behavior.py:
- Rot-bait PR/issue numbers removed from module docstring (#43).
- _TOP_LEVEL_ELEMENT_IDS + import-time drift check replaces the
"refresh this manually" comment (#55).
- _assert_clean_init helper called at the top of every test so init
failures surface as init errors, not as misleading
"side effect didn't fire" failures (#33).
- 4xx restartBtn assertion now reads disabled state via JS and snaps
to body.dataset instead of OR-shortcircuiting against a wiped DOM (#27).
- New test_script_boots_without_broadcastchannel_global covers the
null-guard branch (#15).
- Assertion-restating comments removed (#60).
test_astro_setup_js_behavior.py:
- Rot-bait #1422 reference removed from module docstring (#44).
- _section_has_hidden_class replaces fragile substring slicing (#6).
- test_initial_state_only_client_section_visible now asserts on the
promised visibility, not just absence of errors (#26).
- Per-client smoke now captures config-output text AND instructions
HTML into body.dataset and asserts on non-empty content, catching a
typo that drops the whole per-client branch (#21).
test_astro_tools_js_behavior.py:
- _card_class helper replaces ±200-char substring slicing (#12).
- test_design_mode_toggle now asserts design-only elements lose
'hidden' class, not just the button label flip (#25).
- New tests cover .filter-btn / .cat-btn / .size-filter-btn /
group-category|file|none / sort-alpha / expand-all wiring (#22, #23,
#24) — the adjacent coverage gaps issue #1422 didn't name but that
fit the harness's same regression-class.
test_consent_form_js_behavior.py:
- _build_form_dom docstring fixed (said "three", listed four) (#54).
test_rendered_scripts_parse.py:
- Missing-dependency skip flips to fail when CI=true so a workflow
drift that drops the install step doesn't silently lose parse
coverage (#29).
- Subsumed-test-class reference removed from module docstring.
AGENTS.md:
- "60s probe windows take milliseconds" wording fixed; time-faked
surface documented (#13).
- Per-surface module naming guidance updated; reflects actual files (#10).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): seed data-transports on wizard tiles; add NODE/ESBUILD env overrides; FakeBroadcastChannel ctor guard
CI surfaced a real test-fixture bug exposed by the new jsdomError →
errors routing: setup.astro's connection-click handler reads
`card.dataset.transports` via JSON.parse, but the wizard DOM stubs
were emitting `<button data-client="...">` without the matching
`data-transports` attribute. JSON.parse(undefined) threw "undefined is
not valid JSON" on the jsdomError channel, which the previous
silent-handling code dropped — now correctly surfaced as a test
failure. Fix: serialise the real `transports` array from the
clientsData entry onto each tile.
Also addressing the items previously marked deferred / skipped during
the Gemini + pr-review-toolkit triage:
- NODE_BINARY env override (Gemini #3): _node_binary() helper checks
the env var before falling back to PATH-resolved `node`. Default
unchanged.
- ESBUILD_BINARY env override (Gemini #8): _esbuild_binary() returns
the env-var path when set, else the project-local install. Default
unchanged so the lockfile-pinned install stays the reproducible
default.
- FakeBroadcastChannel constructor guard (sf-hunter #L1): wraps the
`new` in try/catch and records construction failures into errors
before re-raising.
- Trim TestWizardStateMachine class docstring (comment-analyzer #50).
- Tighten the info-call enumeration comment in the 5xx test to
describe the harness's "last entry sticks" semantics rather than
pinning a specific call count (comment-analyzer #48).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): stub layout-dependent JSDOM APIs (scrollIntoView, scrollTo, matchMedia)
The new timer-callback-error routing surfaced a real JSDOM limitation:
`section.scrollIntoView()` (called from the wizard's `scrollToSection`
helper inside a setTimeout) is not implemented in JSDOM. Every
per-client setup-flow test failed with
``timer callback: TypeError: section.scrollIntoView is not a function``
— production behaviour is fine, but the harness's noise filter wasn't
distinguishing real script bugs from JSDOM-missing-API noise.
Adds a defensive no-op stub for scrollIntoView (Element + HTMLElement
prototypes), scrollTo on window, and matchMedia — the three most
common layout-dependent APIs production UI scripts touch. Future
rendered scripts that lean on other layout APIs (IntersectionObserver,
etc.) can extend the list when needed.
Also relaxes the per-client smoke's bare `assert not result.errors` to
rely on `_assert_clean_init` (init/transpile/invoke/jsdom errors) plus
the content-shape assertion. Timer-callback errors from missing JSDOM
APIs are noise; the content-shape check still catches the regression
class the test is named for.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(test): seed .tool-chevron on tool-card fixture for tools.astro expand-all test
The expand-all handler queries `card.querySelector('.tool-chevron')!`
(TypeScript non-null assertion). The runtime `!` doesn't actually
check; chevron is null in the test DOM and `chevron.classList.add(...)`
throws. Production cards include the chevron; our fixture didn't.
Add it alongside `.tool-details` in `_build_tools_dom`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: kingpanther13 <kingpanther13@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 13afa9d commit f7d1b5a
15 files changed
Lines changed: 4056 additions & 62 deletions
File tree
- .github/workflows
- tests
- js
- src/unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
131 | | - | |
132 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
133 | 155 | | |
134 | 156 | | |
135 | 157 | | |
136 | 158 | | |
137 | 159 | | |
138 | | - | |
| 160 | + | |
139 | 161 | | |
140 | 162 | | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
141 | 178 | | |
142 | 179 | | |
143 | 180 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
543 | 543 | | |
544 | 544 | | |
545 | 545 | | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
546 | 610 | | |
547 | 611 | | |
548 | 612 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
0 commit comments