Commit 6f9ca4f
authored
test(e2e): add A2 negative-input tests for optional-id list-or-detail tools (#1058)
* test(e2e): add A2 negative-input tests for optional-id list-or-detail tools
Closes the A2 (optional ID → list or detail) negative-input test gaps from
#914. Two existing soft tests are hardened with explicit error-code and
message-substring assertions; three new tests are added for tools that had
no negative-input coverage on the single-ID lookup branch.
- Hardened test_get_blueprint_not_found in blueprints/test_blueprints.py —
added error.code == 'RESOURCE_NOT_FOUND' assertion alongside the existing
suggestions check.
- Hardened test_get_nonexistent_label in config/test_label_crud.py —
added error.code == 'ENTITY_NOT_FOUND' assertion plus message-substring
check on top of the existing success check.
- New TestGetIntegrationNegativeInputs in integrations/test_list_integrations.py
with test_get_integration_nonexistent_entry_id. The existing tests in
this file exercise list/query/domain-filter modes (A4-style empty results),
not the entry_id single-entry lookup branch, which raises RESOURCE_NOT_FOUND
via _get_single_entry's 404 catch.
- New TestDeviceGetNegativeInputs in registry/test_device_registry.py with
test_get_device_nonexistent_device_id. Single-device lookup raises
ENTITY_NOT_FOUND when the device_id is absent from the registry list.
- New TestUpdatesGetNegativeInputs in updates/test_updates.py with
test_get_updates_nonexistent_entity_id. _get_update_details fetches the
entity state via REST; a 404 response (HomeAssistantAPIError) is mapped to
ENTITY_NOT_FOUND in the except-Exception branch. Live-probed against a
real HA instance: GET /api/states/update.<nonexistent> returns HTTP 404
with body {"message": "Entity not found."}.
ha_get_zone is intentionally not included — its A2 negative path is
already hard-asserted in zones/test_lifecycle.py. ha_config_get_category,
ha_get_todo, ha_get_entity_exposure and ha_config_get_dashboard were
deliberately scoped out (already-hard delete-side coverage in different
file, validation-pathway distinct from A2-classic, legitimate
default-exposure return shape, and a separate maintainer-mandated PR for
dashboard_id dual-accept respectively).
References #914.
* test(e2e): address Gemini review on A2 tests — structured error access
Addresses 5 medium-priority Gemini Code Assist findings on PR #1058:
- blueprints: drop redundant 'not found' substring assertion (already
validated by call_tool_failure(expected_error=...)); switch to direct
result['error'] access instead of .get() since the helper guarantees
the key.
- label_crud, list_integrations, device_registry, updates: replace
str(data.get('error','')).lower() with data['error']['message'].lower()
for structured field access. Repo precedent: automation/test_lifecycle.py
uses the same pattern.
No behavioral change — same assertions, more robust access path.
* test(e2e): address Gemini follow-up — assert structured suggestion presence
Addresses 4 medium-priority Gemini Code Assist findings on PR #1058
re-review:
- label_crud, list_integrations, device_registry: add 'suggestion' or
'suggestions' presence assertion to verify the styleguide.md progressive
disclosure principle (error responses guide next steps).
- updates: switch from safe_call_tool to MCPAssertions.call_tool_failure
for consistency with the file's existing test style; add the same
suggestion presence assertion.
Note on the assertion shape: create_error_response (errors.py:243-248)
sets the singular 'suggestion' key for any non-empty suggestions list and
the plural 'suggestions' key only when len > 1. The four targeted tools
each provide exactly one suggestion in their not-found path, so a strict
'suggestions' (plural) assertion would fail; the disjunction matches both
forms and is robust against future single->multi migrations.
* test(e2e): align blueprint suggestion assertion with disjunction pattern
Addresses Gemini follow-up finding on test_blueprints.py:188.
The plural-only 'suggestions' check is fragile — create_error_response
(errors.py:243-248) only sets the plural key when len > 1. ha_get_blueprint
currently provides two suggestions in its not-found path (blueprints.py:147)
so the plural assertion passes today, but the test would silently break if
a future change reduces the count to one.
Switch to the same 'suggestion' or 'suggestions' disjunction pattern used
in the other four negative-input tests for consistency and robustness.
* test(e2e): align docstring with current assertion shape
Drift fix: docstring previously claimed 'message-substring assertions'
which was removed earlier in this PR (redundant to call_tool_failure's
expected_error parameter). Update wording to reflect the actual
hardening — error-code assertion plus structured suggestion-presence
disjunction (suggestion or suggestions).
* test(e2e): simplify suggestion assertions to singular key
Addresses 4 medium-priority Gemini findings — the 'suggestion' or
'suggestions' disjunction is redundant. create_error_response
(errors.py:243-248) sets the singular 'suggestion' key unconditionally
when any suggestions are provided; the plural 'suggestions' key is
only set additionally when len > 1. Checking the singular key is both
sufficient and aligned with the helper's contract.
Applied uniformly across all 5 tests for consistency.1 parent 0950652 commit 6f9ca4f
5 files changed
Lines changed: 173 additions & 8 deletions
File tree
- tests/src/e2e/workflows
- blueprints
- config
- integrations
- registry
- updates
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
163 | | - | |
| 163 | + | |
| 164 | + | |
164 | 165 | | |
165 | | - | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
166 | 172 | | |
167 | 173 | | |
168 | 174 | | |
169 | 175 | | |
170 | 176 | | |
171 | 177 | | |
172 | 178 | | |
173 | | - | |
| 179 | + | |
174 | 180 | | |
175 | 181 | | |
176 | 182 | | |
177 | | - | |
178 | | - | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
179 | 189 | | |
180 | 190 | | |
181 | 191 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
202 | 202 | | |
203 | 203 | | |
204 | 204 | | |
205 | | - | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
206 | 217 | | |
207 | 218 | | |
208 | 219 | | |
209 | 220 | | |
210 | 221 | | |
211 | | - | |
| 222 | + | |
212 | 223 | | |
213 | 224 | | |
214 | 225 | | |
215 | 226 | | |
216 | 227 | | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
217 | 238 | | |
218 | 239 | | |
219 | 240 | | |
| |||
Lines changed: 47 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| |||
421 | 421 | | |
422 | 422 | | |
423 | 423 | | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
625 | 625 | | |
626 | 626 | | |
627 | 627 | | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
534 | 534 | | |
535 | 535 | | |
536 | 536 | | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
0 commit comments