Skip to content

fix: parallelize deep_search Tier 3 config fetches (closes #879) - #882

Merged
kingpanther13 merged 6 commits into
homeassistant-ai:masterfrom
kingpanther13:fix/deep-search-tier3-parallel
Apr 7, 2026
Merged

fix: parallelize deep_search Tier 3 config fetches (closes #879)#882
kingpanther13 merged 6 commits into
homeassistant-ai:masterfrom
kingpanther13:fix/deep-search-tier3-parallel

Conversation

@kingpanther13

Copy link
Copy Markdown
Member

What does this PR do?

Fixes ha_deep_search missing automations/scripts when bulk config fetch fails (Tier 1/2) and the Tier 3 fallback times out before fetching all configs.

Root cause: Tier 3 fetched configs sequentially, prioritized by name-match score. Automations whose names didn't match the query were deprioritized and skipped when the time budget expired — even if the entity was referenced inside their conditions/actions.

Fix (3 changes):

  1. Parallel batch fetching — Replace sequential for loop with asyncio.gather batches of 10. This fetches 10x more configs in the same time window.

  2. Remove name-score prioritization for fetch order — All configs are now fetched without priority ordering. Name score is still used for result ranking after configs are retrieved. Deep search's purpose is to find matches inside configs, so deprioritizing by name defeats the purpose.

  3. Configurable time budgets — Budgets are now configurable via HAMCP_AUTOMATION_CONFIG_TIME_BUDGET (default: 30s, was 15s) and HAMCP_SCRIPT_CONFIG_TIME_BUDGET (default: 20s, was 10s). Higher defaults are safe since parallel fetching completes faster.

Additional: Logs a warning with fetch/skip counts when the budget is exhausted, so users can diagnose incomplete results.

Closes #879

Type of change

  • 🐛 Bug fix
  • ✨ New feature
  • 📚 Documentation
  • 🔧 Maintenance/refactor
  • 💥 Breaking change

Testing

  • I have tested these changes with a LLM agent
  • All automated tests pass (uv run pytest)
  • Code follows style guidelines (uv run ruff check)

Checklist

  • I have updated documentation if needed

…ore prioritization

When bulk config fetches fail (Tier 1/2), Tier 3 now fetches individual
automation/script configs in parallel batches of 10 instead of sequentially.
Crucially, fetch order is no longer prioritized by name-match score — all
configs are fetched regardless of whether the automation name matches the
query. This ensures entities referenced only inside conditions/actions
(not in the automation name) are found by deep_search.

Changes:
- Replace sequential Tier 3 fetch with parallel asyncio.gather batches
- Remove name-score prioritization for fetch order (score used only for
  result ranking after configs are fetched)
- Make time budgets configurable via HAMCP_AUTOMATION_CONFIG_TIME_BUDGET
  and HAMCP_SCRIPT_CONFIG_TIME_BUDGET env vars
- Increase default budgets (15s→30s automation, 10s→20s script) since
  parallel fetching covers more ground per second
- Log warning when budget is exhausted with fetch/skip counts

Closes homeassistant-ai#879

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where automations and scripts were missing from deep search results when bulk configuration fetching failed. By switching to parallel batch processing and removing restrictive name-based prioritization, the system can now retrieve more configurations within the allotted time budget. Additionally, the inclusion of configurable time budgets and improved logging provides better reliability and diagnostic capabilities for users.

Highlights

  • Parallel Batch Fetching: Replaced sequential configuration fetching with asyncio.gather batches of 10 to significantly improve performance during Tier 3 fallback.
  • Removed Name-Score Prioritization: Eliminated name-based prioritization for fetch order to ensure all configurations are retrieved, allowing deep search to find matches within conditions and actions regardless of the entity name.
  • Configurable Time Budgets: Introduced environment variables (HAMCP_AUTOMATION_CONFIG_TIME_BUDGET and HAMCP_SCRIPT_CONFIG_TIME_BUDGET) to allow users to adjust time budgets for configuration fetching.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the deep_search functionality to fetch automation and script configurations in parallel batches, ensuring that matches within conditions and actions are identified even if the entity name does not match the query. It also allows the configuration of fetch time budgets via environment variables. Feedback includes a correction for a unit test where the mocked time budget is too high to verify budget exhaustion and a recommendation to add error handling when parsing environment variables to prevent potential crashes from invalid input.

Comment thread tests/src/unit/test_deep_search_tier3_parallel.py
Comment thread src/ha_mcp/tools/smart_search.py Outdated
kingpanther13 and others added 2 commits April 5, 2026 18:55
…dict

The asyncio.gather call in Tier 3 was assigned to `results`, overwriting
the outer `results` dict that holds search results by category. Renamed
to `batch_results` to avoid the collision. Also fixed test to use
_make_tools pattern for proper settings mocking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…timing

- Wrap env var float parsing in try/except to prevent startup crash on
  invalid values (Gemini review feedback)
- Fix test_tier3_respects_time_budget: increase per-fetch sleep from 0.5s
  to 1.5s so parallel batches actually exceed the 2s budget (batches of 10
  run concurrently, so 0.5s per fetch = 0.5s per batch < 2s budget)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@kingpanther13 kingpanther13 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed both in 19f4a1e: test sleep increased to 1.5s so batch execution exceeds budget, and env var parsing wrapped in try/except with fallback to defaults.

@kingpanther13
kingpanther13 marked this pull request as ready for review April 6, 2026 01:07
@kingpanther13
kingpanther13 requested a review from a team April 6, 2026 01:07
@kingpanther13
kingpanther13 enabled auto-merge (squash) April 6, 2026 01:07
@Patch76

Patch76 commented Apr 6, 2026

Copy link
Copy Markdown
Member
Both automated review findings were valid and well-addressed in `19f4a1e5` — safe env
var parsing via `_env_float` and the sleep bump to `1.5s` so the budget window is
actually exceeded between batches. Nicely done.

One small thing I noticed while reading through the test:

The comment on `test_tier3_respects_time_budget` describes the expected behavior as
_"first batch takes ~1.5s, second batch would exceed budget"_ — implying a full first
batch of 10 items completes before the budget check kicks in. I traced through the
asyncio timing manually:

- Batch 1 starts at t=0, completes at ~1.5s → still under the 2s budget → Batch 2 launches
- Batch 2 completes at ~3.0s → budget exceeded → Batch 3 skipped
- Result: `call_count = 20`

The lower bound `assert call_count > 0` is technically true here, but it would also
pass if only a single item had been fetched for some other reason. Would
`assert call_count >= 10` be a closer match to what the comment is actually describing?

Just a thought — feel free to ignore if you had a reason for keeping it loose.

Assert call_count >= 10 (one full batch) instead of > 0. The comment
describes expecting at least one complete batch, so the assertion should
match. Thanks @Patch76.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kingpanther13

Copy link
Copy Markdown
Member Author

@Patch76 Good catch — you're absolutely right. The lower bound > 0 would pass even if only a single item was fetched, which doesn't match the comment's claim about completing at least one full batch.

Fixed in c5e7874: changed to assert call_count >= 10 with an updated comment describing the expected timing (batch 1 at t=0→1.5s completes, batch 2 may start, batch 3 skipped). Thanks for tracing through the asyncio timing.

@Patch76

Patch76 commented Apr 6, 2026

Copy link
Copy Markdown
Member

LGTM 🚀

Comment thread src/ha_mcp/tools/smart_search.py Outdated
Comment thread src/ha_mcp/tools/smart_search.py
Comment thread tests/src/unit/test_deep_search_tier3_parallel.py Outdated
Comment thread src/ha_mcp/tools/smart_search.py Outdated
Comment thread src/ha_mcp/tools/smart_search.py Outdated
Comment thread src/ha_mcp/tools/smart_search.py
Comment thread tests/src/unit/test_deep_search_tier3_parallel.py Outdated
Comment thread tests/src/unit/test_deep_search_tier3_parallel.py Outdated
1. Remove return_exceptions=True — inner functions already handle errors
2. _env_float: log warning on invalid values, fix os.environ.get pattern
3. Add script-path test (TestAttemptCScriptParallelFetch)
4. Fix misleading "Fetch ALL" comment — now says "subject to time budget"
5. Rename Tier 3 → Attempt C for consistency with inline code comments
6. Track failed_count separately in budget warning messages
7. Replace fragile 1.5s sleep with 0.01s + proportional budget (0.005s)
8. Fix imprecise assertion comments — remove hedging language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@sergeykad sergeykad left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 8 review items addressed. Code changes verified, CI green (Performance Tests failure is unrelated Docker rate-limiting).

@kingpanther13
kingpanther13 merged commit c11d994 into homeassistant-ai:master Apr 7, 2026
14 of 15 checks passed
@github-actions

github-actions Bot commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

🧪 Your changes are now in the dev channel!

Your PR has been merged to master and is available for testing in the dev channel.

Test your changes before the next stable release (biweekly Wednesday):
📖 Dev Channel Documentation

Quick start

# Run dev version
uvx ha-mcp-dev

# Check version
uvx ha-mcp-dev --version

Docker:

docker pull ghcr.io/homeassistant-ai/ha-mcp:dev
docker run --rm -i \
  -e HOMEASSISTANT_URL=http://your-ha:8123 \
  -e HOMEASSISTANT_TOKEN=your_token \
  ghcr.io/homeassistant-ai/ha-mcp:dev

Found an issue? Please open a new bug report and mention this PR for context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

deep_search misses automations when bulk config fetch fails and Tier 3 fallback times out

3 participants