feat: Search-based tool discovery with categorized call proxies

[kingpanther13]

What does this PR do?

Adds an opt-in CategorizedSearchTransform (ENABLE_TOOL_SEARCH=true) that replaces the full 89+ tool catalog with a unified BM25 search tool and three categorized call proxies (read/write/delete). Reduces idle context from ~46K tokens to ~2-3K tokens while maintaining full functionality and client compatibility.

How it works:

• search_tools — unified BM25 search across all tools (results include full annotations so the LLM knows read vs write vs delete)
• call_read_tool — readOnlyHint: true, clients can auto-approve
• call_write_tool — destructiveHint: true, requires confirmation
• call_delete_tool — destructiveHint: true, requires confirmation

Pinned tools (always visible for individual permission gating):
ha_restart, ha_reload_core, ha_backup_create, ha_backup_restore, ha_get_overview, ha_report_issue, and list_resources/read_resource (when ENABLE_SKILLS_AS_TOOLS=true)

Bootstrap for LLM awareness:

• Server instructions updated with tool discovery workflow when enabled
• ha_get_overview response includes tool_discovery section with hints and pinned tool list
• Search tool description lists all categories and instructs the LLM to search first

BM25 search quality tuning (SearchKeywordsTransform):

• A lightweight FastMCP Transform subclass that appends extra keywords to specific tool descriptions so BM25 ranks them correctly for common queries
• Only active when ENABLE_TOOL_SEARCH=true — original tool docstrings are unchanged
• Fixes three ranking issues found during search quality testing:
  • "find entities" → ha_search_entities ranks above ha_deep_search
  • "get/read automation" → ha_config_get_automation ranks above ha_config_set_automation
  • "create helper" → ha_config_set_helper ranks above ha_config_remove_helper
• To tune rankings for additional tools, add an entry to the _SEARCH_KEYWORDS dict in server.py — no other files need changing

Subclasses FastMCP 3.1's BM25SearchTransform (~200 lines). Tools auto-categorized by existing MCP annotations. Works with ANY MCP client. Pure opt-in — disable the flag and everything works exactly as before.

Related: #637 (category gateways approach), #726 (CodeMode sandbox)

Type of change

• 🐛 Bug fix
• ✨ New feature
• 📚 Documentation
• 🔧 Maintenance/refactor
• 💥 Breaking change

Testing

• I have tested these changes with a LLM agent
• All automated tests pass (uv run pytest)
• Code follows style guidelines (uv run ruff check)

Checklist

• I have updated documentation if needed

🤖 Generated with Claude Code

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new, opt-in search-based tool discovery system designed to optimize LLM interactions. By replacing a large, static tool catalog with a dynamic search mechanism and categorized call proxies, the system drastically reduces the token footprint in the LLM's context. This enhancement improves efficiency and scalability without compromising the agent's access to the full range of functionalities, while also providing clear guidance for LLMs on how to discover and utilize tools based on their operational intent (read, write, delete).

Highlights

• Search-based Tool Discovery: Introduced an opt-in CategorizedSearchTransform that replaces the full tool catalog with a unified BM25 search tool and three categorized call proxies (read/write/delete).
• Token Usage Reduction: Significantly reduced idle context token usage for LLMs from approximately 46K to 2-3K, while maintaining full functionality and client compatibility.
• LLM Workflow Guidance: Updated server instructions and the ha_get_overview response to include a new tool discovery workflow, guiding LLMs on how to find and use tools effectively.
• Categorized Call Proxies: Implemented call_read_tool, call_write_tool, and call_delete_tool proxies, each with specific MCP annotations (readOnlyHint, destructiveHint) to enable client-side permission handling.
• Pinned Tools: Defined a set of critical tools (e.g., ha_restart, ha_backup_create) that remain directly visible, bypassing the search mechanism for individual permission gating.

Changelog

• src/ha_mcp/config.py
  • Added a new enable_tool_search boolean setting to control the activation of the tool discovery feature.
• src/ha_mcp/server.py
  • Integrated a new _apply_tool_search method into the server initialization to apply the search transform.
  • Enhanced _build_skills_instructions to append detailed workflow instructions for LLMs when tool search is enabled.
  • Defined _PINNED_TOOLS and _SEARCH_TOOL_DESCRIPTION constants to support the categorized search transform.
  • Implemented the _apply_tool_search method to add the CategorizedSearchTransform to the MCP server, including logic for pinning specific tools.
• src/ha_mcp/tools/tools_search.py
  • Modified the ha_get_overview function to conditionally include a tool_discovery section with hints and a list of pinned tools when the tool search feature is active.
• src/ha_mcp/transforms/init.py
  • Exported the new CategorizedSearchTransform class, making it available for import.
• src/ha_mcp/transforms/categorized_search.py
  • Introduced the CategorizedSearchTransform class, extending BM25SearchTransform to implement categorized tool search.
  • Implemented the _categorize_tool helper function to classify tools as read, write, or delete based on their annotations and naming conventions.
  • Created _make_categorized_proxy to generate specialized call proxies (call_read_tool, call_write_tool, call_delete_tool) with appropriate MCP annotations.
  • Overrode the transform_tools method to replace the default tool listing with the unified search tool, pinned tools, and the new categorized call proxies.

Activity

• No specific activity (comments, reviews, or progress updates) has been provided in the context for this pull request.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a valuable feature for search-based tool discovery, which significantly reduces token usage. The implementation using a FastMCP transform is clean and well-integrated. The code is generally of high quality. I've identified a few areas for improvement related to performance, maintainability, correctness, and adherence to the project's style guide. These suggestions aim to make the new feature more robust and efficient.

[kingpanther13]

@julienld @sergeykad I haven't done BAT testing yet but I Have tested it directly in claude.ai and it works pretty well., Since this is an opt-in feature I Feel that it can be merged. I did slightly modify ha_get_states to make the search function work better (if it searched for "lights status" it couldn't find this tool for example) but literally only added 5 words. I'm sure more little edge cases like that will pop up with this but overall this is going to save a TON of tokens and potentailly be a lot faster to use too.

[kingpanther13]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces an excellent feature for search-based tool discovery, which will significantly reduce token usage for LLM clients. The implementation is well-structured, leveraging a transform to dynamically alter the tool catalog. My review focuses on improving maintainability by reducing code duplication, enhancing correctness by avoiding hardcoded values, and ensuring adherence to the project's error handling guidelines.

[sergeykad]

UAT results — master vs #727 (ENABLE_TOOL_SEARCH=true)

Ran all 13 UAT stories against both branches using the openai agent with a local model via LM Studio.

Test setup:

• Model: qwen3.5-9b-claude-4.6-opus-distilled-uncensored-v2 (9B parameter local model, no native deferred tools)
• Runner: openai agent (OpenAI-compatible API, no Claude-specific tool caching or deferred tool support)
• master fa925fc — standard tool catalog (~89 tools)
• PR c283cfa — ENABLE_TOOL_SEARCH=true (search + 4 proxy tools + pinned tools)
• Fresh HA container per run, isolated state per story
• tokens_first_input = prompt tokens on the first LLM call only — the cleanest measure of idle context size before any tool results accumulate

Story	Title	master (fa925fc)	#727 search (c283cfa)
s01	Create sunset automation	✅ 89s 39,025tok 2calls	✅ 27s 5,493tok 1calls
s02	Motion-activated light	✅ 66s 39,062tok 3calls	✅ 28s 5,530tok 2calls
s03	Debug automation not firing	✅ 64s 39,030tok 6calls	❌ 72s 5,498tok 11calls
s04	Create room dashboard	✅ 70s 39,047tok 3calls	❌ 6s — 0calls
s05	Discover entities	✅ 72s 39,027tok 7calls	✅ 39s 5,495tok 1calls
s06	Goodnight routine script	✅ 68s 39,044tok 4calls	❌ 116s 5,512tok 20calls
s07	Update automation + condition	❌ 133s 39,055tok 20calls	✅ 34s 5,523tok 6calls
s08	Complex multi-condition auto	✅ 129s 39,092tok 12calls	✅ 24s 5,560tok 2calls
s09	Vacation mode toggle	❌ 151s 39,057tok 20calls	✅ 29s 5,525tok 5calls
s10	Create areas + organize	✅ 61s 39,041tok 4calls	✅ 34s 5,509tok 6calls
s11	History analysis	❌ 102s 39,056tok 7calls	❌ 43s 5,524tok 5calls
s12	Create + assign labels	✅ 65s 39,059tok 4calls	✅ 31s 5,527tok 5calls
s13	Find automations by entity	✅ 86s 38,994tok 8calls	✅ 13s 5,462tok 2calls
TOTAL (all 13)		10/13 pass	9/13 pass
TOTAL (both passed, 7 stories)	s01 s02 s05 s08 s10 s12 s13	568s 39,042tok avg 40calls	196s (−65%) 5,510tok avg (−86%) 19calls (−52%)

Notes:

• Aggregate numbers are compared only on stories both branches passed (7/13), to avoid skew from stories where one branch hit the 20-call limit or a model crash.
• First-turn context is ~39K on master vs ~5.5K on the PR branch. Higher than the claimed 2–3K because this model generates more verbose tool schema output; the reduction ratio is consistent with the PR's claim.
• Pass rate is comparable (10/13 vs 9/13) and the failures are largely on different stories — the PR fixed s07 and s09 which master fails, but regressed s03 and s06.
  • s03/s06 regressions appear to be the model struggling with the search-first workflow (s06 hit the 20-call limit rather than giving up), not functional gaps in the tools.
  • s04: model crashed (exit 18) before executing any tool calls on the PR branch — not a bug in the PR, likely a schema/context issue specific to this model.
• s11 (history analysis) fails on both branches with this model.

---

The context reduction numbers check out and the pass rate is comparable for a 9B local model. The implementation looks good to me.

[sergeykad]

One issue cannot be placed inline because the relevant lines are not in a diff hunk:

ha_deep_search docstring says default 20, code says 5 (src/ha_mcp/tools/tools_search.py:658)

The docstring at line 658 says limit: Maximum total results to return (default: 20), but the function signature at line 632 has limit: int = 5. A caller reading the docstring will expect 20 results by default but receive 5. Fix the docstring to match the actual default.

[kingpanther13]

Re: ha_deep_search docstring default mismatch — fixed: docstring now says default: 5 matching the actual function signature.

[sergeykad]

Documentation is incomplete for ENABLE_TOOL_SEARCH. The following gaps need to be addressed before merge:

1. Production add-on config missing the option
homeassistant-addon/config.yaml doesn't include enable_tool_search in either options: or schema:. The dev config has it, but the production config doesn't — HA add-on users have no way to toggle this from the UI. Compare how enable_skills and enable_skills_as_tools are declared:

options:
  enable_tool_search: false
schema:
  enable_tool_search: bool?

2. Production add-on translation missing
homeassistant-addon/translations/en.yaml needs the same entry that exists in the dev translation.

3. .env.example not updated
New users running via Docker/stdio won't discover this option. Add a commented entry with a brief note on when to enable it.

4. config.py field has no description
enable_tool_search: bool = Field(False, alias="ENABLE_TOOL_SEARCH") — add a description= parameter consistent with how other fields in the same class are documented.

5. No README/docs entry
The docs should explain when to enable it. The flag has two benefits: it reduces visible tool count (important for clients that cap total tools, e.g. at 100), and for models without deferred tool support or with limited context windows it cuts ~46K tokens of idle context. With deferred tool support (e.g. Claude), the full catalog never enters context — the flag adds no token benefit there, and disabled is the right default.

[kingpanther13]

Addressed 4 of 5 items in 7aceb08:

1. Production config.yaml — Added enable_tool_search: false to options: and enable_tool_search: bool? to schema:, matching how enable_skills and enable_skills_as_tools were added simultaneously to both configs in #679.

2. Production translations — Added matching enable_tool_search entry to homeassistant-addon/translations/en.yaml.

3. .env.example — Added commented ENABLE_TOOL_SEARCH=false entry with a brief note on when to enable it, for Docker/stdio users.

4. config.py Field description= — Pushing back on this one. None of the 17 existing Field() definitions in Settings use description= — all use inline comments above the field instead. The PR already has a 3-line comment:

# Tool search transform — replaces the full tool catalog with a unified
# BM25 search tool and categorized call proxies (read/write/delete).
# Dramatically reduces idle context token usage for LLMs.
enable_tool_search: bool = Field(False, alias="ENABLE_TOOL_SEARCH")

User-facing descriptions live in translations/en.yaml (now added to both configs). Adding description= to just this field would be inconsistent with the rest of the class.

5. DOCS.md — Added enable_tool_search section to Configuration Options with guidance on when to enable (models without deferred tool support, limited context windows, tool-count caps) and when to leave disabled (Claude/deferred tool clients).

Concerns addressed

[github-actions]

🧪 Your changes are now in the dev channel!

Your PR has been merged to master and is available for testing in the dev channel.

Test your changes before the next stable release (biweekly Wednesday):
📖 Dev Channel Documentation

Quick start

# Run dev version
uvx ha-mcp-dev

# Check version
uvx ha-mcp-dev --version

Docker:

docker pull ghcr.io/homeassistant-ai/ha-mcp:dev
docker run --rm -i \
  -e HOMEASSISTANT_URL=http://your-ha:8123 \
  -e HOMEASSISTANT_TOKEN=your_token \
  ghcr.io/homeassistant-ai/ha-mcp:dev

---

Found an issue? Please open a new bug report and mention this PR for context.