feat: tool search proxy for context reduction (phase 1) - #627
feat: tool search proxy for context reduction (phase 1)#627kingpanther13 wants to merge 13 commits into
Conversation
Implements the server-side Tool Search Proxy pattern recommended by Anthropic (code execution with MCP blog) and validated by Speakeasy (100x token reduction). Instead of registering all tools with MCP, niche tools are served through 3 meta-tools: ha_find_tools, ha_get_tool_details, ha_execute_tool. Phase 1 proxies 10 tools from 5 modules (zones, labels, addons, voice_assistant, traces). Net tool count decreases by 7 (10 removed, 3 meta-tools added). Tool descriptions and implementations are completely unchanged — they're stored server-side and returned on demand. Schema enforcement via required tool_schema hash prevents LLMs from calling tools without reading documentation first. Migration roadmap in tool_proxy.py PROXY_MODULES comments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary of ChangesHello @kingpanther13, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant architectural change to optimize LLM interaction with Home Assistant tools. By implementing a server-side tool search proxy, it addresses the critical issue of excessive context token consumption caused by registering all tool definitions upfront. This new pattern allows for dynamic tool discovery and execution through a set of lightweight meta-tools, drastically reducing the LLM's idle context footprint and enabling compatibility with models that have stricter token limits. This initial phase proxies 10 tools, laying the groundwork for a complete migration. Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a significant architectural improvement with the tool search proxy pattern, which will drastically reduce token usage. The implementation is well-thought-out and includes a good set of unit tests for the new proxy logic.
My review has identified a few key areas for improvement:
- Error Handling: The new meta-tools currently use custom dictionary-based error returns. The repository style guide requires using the standardized structured error helpers from
errors.pyto ensure consistency. This is a high-severity issue. - Testing: While the unit tests are comprehensive, the style guide also requires E2E tests for new MCP tools. Adding automated E2E tests for these foundational meta-tools is critical and is flagged as a high-severity issue.
- Code Style: There are minor code style issues, such as local imports within functions, which should be moved to the top of the file for better readability and adherence to PEP 8.
- Fix ruff lint: I001 import sorting, SIM300 Yoda condition, PERF401 list.extend - Use structured error helpers from errors.py in all meta-tool error paths (Gemini HIGH: create_resource_not_found_error, create_validation_error, etc.) - Move hashlib/typing/types imports to top of file (Gemini MEDIUM: PEP 8) - Update E2E tests to route proxied tools through ha_execute_tool - Add proxy_helpers.py utility for transparent proxy routing in E2E tests - Add E2E tests for meta-tools (Gemini HIGH: ha_find_tools, ha_get_tool_details, ha_execute_tool) covering search, details, execution, and error rejection - Update unit test assertions for structured error format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update test_label_operations.py and test_lifecycle.py (labels) to use proxy_call_tool for ha_config_set/get/remove_label - Update test_voice_assistant.py to use proxy_call_tool for ha_get_entity_exposure - Update test_entity_rename.py to use proxy_call_tool for ha_get_entity_exposure - Update test_lifecycle.py (zones) to use ProxyMCPAssertions for ha_get/create/update/delete_zone - Make proxy_call_tool transparent: falls back to direct call for non-proxied tools, so it works as drop-in replacement - Add ProxyMCPAssertions class for tests using MCPAssertions pattern with proxied tools Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…test The valid_assistants field is stringified inside error.message when routed through the tool proxy, so check both top-level and error string. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the proxy-aware tool routing from individual test files into the mcp_client fixture in conftest.py. This eliminates all modifications to existing test files and means future proxy phases require zero test changes — just add module names to PROXY_MODULES. - Delete proxy_helpers.py (143 lines) - Revert 7 test files to master (labels, zones, traces, voice, rename) - Add _ProxyAwareClient wrapper in conftest.py with error unwrapping - Trim tool_proxy.py docstrings and roadmap comments (-64 lines) Previous state tagged as pre-trim-v1 for easy revert. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Unit tests: 365 -> 224 lines (DRY fixture, merged similar tests) - E2E tests: 265 -> 133 lines (helper method, removed logger noise) - ha_execute_tool: simplified args parsing and error messages Same test coverage, same assertions, just concise. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Let ToolError propagate through proxy instead of wrapping it, so FastMCP handles isError flag correctly (fixes voice assistant test) - Remove trailing whitespace on blank lines in test_traces.py (W293) - Fix Yoda condition in test_tool_proxy.py (SIM300) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a well-designed tool search proxy to significantly reduce token usage, which is a major architectural improvement. The implementation is robust, leveraging a mock MCP for non-invasive tool discovery and providing a transparent proxy for E2E tests. The code is well-documented and includes comprehensive unit and E2E tests. I have a few suggestions to improve maintainability and fix a couple of minor issues.
- Collapse verbose if/continue search chain into single boolean (tool_proxy.py)
- Remove unnecessary isinstance(params, dict) guards in _schema_hash,
_make_summary, and _format_parameters — register_tool() guarantees dict
- Replace fragile startswith("ha_find_tool") with _META_TOOLS constant set
- Remove dead _unwrap_proxy_error method (ToolError now propagates directly)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This is an excellent pull request that introduces a powerful pattern for managing a large number of tools, significantly reducing token usage. The implementation of the tool proxy with meta-tools (ha_find_tools, ha_get_tool_details, ha_execute_tool) is well-structured and follows best practices. The use of a mock MCP to discover tools without registering them is particularly clever. The changes to the test infrastructure, especially the _ProxyAwareClient, ensure that existing tests remain valid with minimal changes, which is a great example of thoughtful engineering. I've identified a bug in the type-to-JSON schema conversion and an opportunity to further improve the generated schemas for array types. Overall, this is a high-quality contribution that will greatly improve the scalability and usability of ha-mcp.
Use origin type for type_map lookup so list[str] -> "array", dict[str, int] -> "object", etc. instead of falling back to "string". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add _get_array_items_schema() to enrich array-type parameters with
proper JSON Schema items (e.g. list[str] -> items: {type: string}).
Handles Union unwrapping for list[str] | None patterns. Needed for
future proxy phases where many tools use list[str] parameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Review: PR #627 — Tool Search Proxy for Context Reduction (Phase 1)What the PR DoesThis PR replaces direct MCP registration of 10 tools (from 5 modules: zones, labels, addons, voice_assistant, traces) with 3 meta-tools:
The goal is to reduce idle context consumption. Today, 94+ tools occupy ~35,500 tokens — 17.8 % of Claude's window and 27.8 % of GPT-4o's. The problem is real and well-quantified. MCP ComplianceThe 3 meta-tools are valid MCP tools: they register normally, appear in 1. Tools vanish from
|
| Dimension | Assessment |
|---|---|
| Problem identification | Strong — real and well-quantified |
| Wire-level MCP compliance | Compliant — meta-tools are valid |
| Design-intent MCP alignment | Misaligned — hides tools, strips annotations, replaces typed invocation |
| Code quality | Good — clean registry, thorough tests |
| Architectural direction | Concerning — parallel discovery protocol diverges from SEP-1821 |
| Breaking changes | Yes — 10 tools disappear from tools/list |
| Security model | Weak — schema hash provides false confidence |
Bottom line: Well-engineered code solving a real problem at the wrong layer. It builds a custom tool-discovery protocol on top of MCP instead of leveraging MCP's own mechanisms. The approach will cause compatibility issues with standard MCP clients and diverges from the direction the spec is heading. The project would be better served by adopting or contributing to protocol-level solutions for tool-set scaling.
|
Thanks for the thorough review, Sergey. A few notes upfront, then I'll address each point. This Is a Draft PRThis PR is marked Draft and was not ready for review. It's a working proof-of-concept for discussion, not a merge candidate. That said, I appreciate the detailed feedback — let me address every concern. Why Only 10 Tools (Phase 1)The phased approach is intentional. We're proxying 5 low-traffic modules (zones, labels, addons, voice_assistant, traces) to validate the pattern works end-to-end — correct schema extraction, error propagation, E2E test transparency — before expanding. If the approach is accepted, expanding to all tools is trivial (add module names to The "Extra Round-Trips" ArgumentThis framing is misleading. Yes, a single proxied tool call takes 2-3 LLM turns instead of 1. But the math overwhelmingly favors the proxy: Without proxy: 94 tools x ~375 tokens each = ~35,000 tokens of idle schema on every single turn, whether or not any tool is called. With proxy (all tools proxied): 3 meta-tools x ~200 tokens = ~600 tokens idle. A find->details->execute flow adds ~1,500 tokens across 3 turns. That's ~600 tokens idle + ~1,500 tokens per actual tool use vs ~35,000 tokens on every turn regardless. In a 20-turn conversation where the user invokes 3-4 tools, the proxy saves roughly 650K tokens over the session. The "extra round-trips" cost is noise compared to the idle context elimination. This is exactly why Anthropic and Speakeasy independently recommend the meta-tool/progressive-disclosure pattern — and why Speakeasy measured 96-160x token reduction with essentially the same architecture. Why
|
| Alternative | Assessment |
|---|---|
listChanged |
Wrong layer, does not solve token bloat, requires server-side omniscience, fragmented client support |
| SEP-1821 | Dormant, no sponsor, rejected predecessor, no implementation timeline |
| Multiple MCP servers | Valid but orthogonal — splits tools but does not reduce total schema tokens across servers |
| Rich annotations | Does not reduce tokens — annotations are additional metadata, not a filtering mechanism |
| Connection-time negotiation | Interesting but no spec support, requires custom client-side implementation |
Bottom Line
This PR implements a pattern that Anthropic recommends, Speakeasy independently validated (96-160x token reduction), and multiple open-source projects have adopted. The "extra round-trips" cost is a rounding error compared to 35K tokens of idle context per turn.
That said — this is a draft, and I am open to any architectural direction the maintainers prefer. If there is a protocol-level solution on the horizon that makes this unnecessary, I am happy to wait for it or contribute to it.
- Add FastMCP SDK compatibility check in _MockMCP.__init__ that warns if the SDK's tool() method is missing, failing fast on breaking changes - Increase schema_hash from 8 to 12 hex chars (48 bits / 281 trillion values) to reduce LLM hallucination risk Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
One thing I should have mentioned above —
I considered building a hybrid — detect The proxy pattern is guaranteed to work with any MCP client. The 3 meta-tools register via standard If/when Also pushed a commit (3665ace) addressing two code-level concerns from the review:
|
Missed the E2E assertion when increasing schema_hash from 8 to 12 chars. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
After doing some LLM testing I've determined that this one would actually work quite well if we converted all of the tools over at once, BUT doing a phased conversion definitely would completely break the addon. So it's all or nothing. I'm leaning toward #616 (if I get it optimized) since it can be phased. However, I think this idea might still be useful when it comes to the thought of consolidating some tools especially lesser used ones, instead of consolidating everything into one mega group. I intend to group together some things using this method, like for example ha_config_set_helper and ha_create_config_entry_helper canCould wrap both behind a single ha_config_set_helper that routes internally based on type. The agent shouldn't need to know the difference between storage vs config-entry helpers. So @julienld can I just completely rework this PR then make it ready for review once I have it figured out, or should I create a whole new PR just using this as a backbone? I think that it would still be able to play nice with #616 too. This would really be best for just reducing tool count drastically so we could add more tools in the future. |
|
Some thoughts: this could be developed as a general tool: a FastMCP tool search proxy or even a FastMCP plugin. It would work with any large MCP or collection of MCPs. It might also benefit from semantic search : BM25, vectorization, ... |
Replace direct MCP registration of lesser-used tools with domain-named
gateway tools that combine discovery and execution in a single step.
Phase 1 implements ha_manage_dashboards, which consolidates 12 dashboard
and resource tools behind one gateway. This reduces idle context tokens
while maintaining full functionality through 1-step access:
- gateway() → lists all sub-tools with full parameter schemas
- gateway(tool="ha_config_set_dashboard", args='{...}') → executes
This approach addresses discoverability issues found in PR homeassistant-ai#627's
3-step meta-tool pattern (ha_find_tools → ha_get_tool_details →
ha_execute_tool), where LLMs failed to discover proxied capabilities.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
I'm going to go ahead and close it for now, opening #637 as a better option with similar concept. |
Replace direct MCP registration of lesser-used tools with domain-named
gateway tools that combine discovery and execution in a single step.
Phase 1 implements ha_manage_dashboards, which consolidates 12 dashboard
and resource tools behind one gateway. This reduces idle context tokens
while maintaining full functionality through 1-step access:
- gateway() → lists all sub-tools with full parameter schemas
- gateway(tool="ha_config_set_dashboard", args='{...}') → executes
This approach addresses discoverability issues found in PR homeassistant-ai#627's
3-step meta-tool pattern (ha_find_tools → ha_get_tool_details →
ha_execute_tool), where LLMs failed to discover proxied capabilities.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace direct MCP registration of lesser-used tools with domain-named
gateway tools that combine discovery and execution in a single step.
Phase 1 implements ha_manage_dashboards, which consolidates 12 dashboard
and resource tools behind one gateway. This reduces idle context tokens
while maintaining full functionality through 1-step access:
- gateway() → lists all sub-tools with full parameter schemas
- gateway(tool="ha_config_set_dashboard", args='{...}') → executes
This approach addresses discoverability issues found in PR homeassistant-ai#627's
3-step meta-tool pattern (ha_find_tools → ha_get_tool_details →
ha_execute_tool), where LLMs failed to discover proxied capabilities.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3 participants
What does this PR do?
Implements the server-side Tool Search Proxy pattern to dramatically reduce idle context token usage. Instead of registering all tools with MCP (consuming ~35K tokens of tool definitions on every turn), niche tools are served through 3 lightweight meta-tools.
This is the same pattern recommended by Anthropic for large tool libraries and independently validated by Speakeasy (100x token reduction on 400-tool servers).
The Problem
ha-mcp v6.6.1 registers 94+ tools. Their combined definitions consume ~35,500 tokens — 17.8% of Claude's context, 27.8% of GPT-4o's, and well over ChatGPT's hard 16K tool token limit (#614). This makes ha-mcp completely unusable on ChatGPT and wastes significant context on every turn regardless of which tools are used.
Why Not
listChanged/ Dynamic Registration?listChangedwas my initial preferred approach — dynamically add/remove tools as the conversation narrows while keeping them first-class. I abandoned it after research showed:listChanged. Many clients and LLMs may not handle dynamic tool list updates correctly or at all.listChangedreduces the server's tool list, but the client still sends full schemas for every visible tool on every turn.listChangedfirst, fall back to proxy), but the complexity was not justified given the uncertain client support landscape.The proxy pattern is guaranteed to work with any MCP client. The 3 meta-tools are standard MCP tools registered via
@mcp.tool()— they appear intools/listwith properinputSchemaand annotations like any other tool. Any client that can call an MCP tool can use them, regardless of which LLM is behind it.The Solution
Three meta-tools replace direct registration for proxied tools:
ha_find_tools(query)ha_get_tool_details(tool_name)ha_execute_tool(tool_name, args, tool_schema)Schema enforcement:
ha_execute_toolrequires atool_schemahash (12 hex chars / 48-bit) that can only come fromha_get_tool_details. This structurally prevents LLMs from calling tools without reading documentation — same principle as theguide_responsepattern from #616 but applied universally.Annotation transparency:
ha_find_toolsreturnsis_destructivefor each result.ha_get_tool_detailsreturns the full annotation set (destructiveHint,readOnlyHint,idempotentHint).ha_execute_toolis annotated asdestructiveHint: Trueas a conservative default.Phase 1 (This PR): 10 Tools from 5 Modules
tools_zonesha_get_zone,ha_create_zone,ha_update_zone,ha_delete_zonetools_labelsha_config_get_label,ha_config_set_label,ha_config_remove_labeltools_addonsha_get_addontools_voice_assistantha_get_entity_exposuretools_tracesha_get_automation_tracesNet result: Tool count decreases by 7 (10 removed, 3 meta-tools added). Phased approach validates schema extraction, error propagation, and E2E test transparency before expanding. Expansion is trivial — add module names to
PROXY_MODULESset.Migration Roadmap
Each subsequent phase is a one-line change — add module names to
PROXY_MODULESset intool_proxy.py. No tool implementation code is modified at any phase.Known Trade-offs
tools/list— this is intentional (the point is to remove idle context), but means MCP clients that previously called these tools directly will need to use the meta-tool workflow. For this ha-mcp server specifically, these tools have never been in a stable release with direct registration.ha_execute_tooltakesargsas a JSON string rather than typed parameters. Server-side validation (schema hash + required param check + JSON parse) mitigates this.What is NOT Changed
ha_get_tool_detailsLLM Testing Plan
Test Prompts
The following prompts should be given to LLM agents connected to ha-mcp without any warning that the tool model has changed.
Test 1 — Tool Discovery (Basic)
Expected: Agent discovers
ha_get_zoneviaha_find_tools, reads details, executes successfully.Test 2 — Destructive Operation (Schema Enforcement)
Expected: Agent discovers
ha_create_zone, gets full schema, passes correct args with schema_hash.Test 3 — Cross-Category Discovery
Expected: Agent finds both
ha_config_get_labelandha_get_entity_exposurein separate searches.Test 4 — Addon Query
Expected: Agent discovers
ha_get_addon, reads details, executes with no args orsource='installed'.Test 5 — Debugging Workflow
Expected: Agent discovers
ha_get_automation_traces, asks for automation entity_id, executes.Test 6 — Unknown Capability
Expected: Agent sees "zone" tools in
ha_find_toolscatalog and proceeds. Should NOT say "I don't have that capability."Test 7 — Direct Tool Access Still Works
Expected: Agent uses
ha_call_servicedirectly (still registered normally). Proxy does not interfere.Target Clients for Testing
References
search_toolspattern, 98.7% token reductionChanges
src/ha_mcp/tools/tool_proxy.py— New: proxy registry, meta-tools, MockMCP with SDK guard, schema enforcementsrc/ha_mcp/tools/registry.py— Modified: skip proxy modules, wire up proxy registrationtests/src/unit/test_tool_proxy.py— New: 26 unit tests covering registry, meta-tools, enforcementtests/src/e2e/conftest.py— Modified:_ProxyAwareClientwrapper for transparent E2E routingType of change
tools/list— accessible via meta-tools)Testing
uv run pytest)uv run ruff check)Checklist