fix: resolve WebSocket race conditions and improve error handling (#378)

* fix: resolve WebSocket race conditions and improve error handling

## Summary
Fixes race conditions from parallel tool calls creating multiple WebSocket connections and improves error handling for common failure scenarios.

## Changes

### WebSocket Race Condition Fix
- **rest_client.py**: Use singleton WebSocket connection instead of ephemeral connections
- Prevents multiple simultaneous handshakes that trigger 403 throttling
- Add automatic retry (2 attempts) for transient 403 errors
- Affects 15+ tools using send_websocket_message

### Error Handling Improvements
- **ha_eval_template**: Detect 403 errors, provide specific suggestions about reverse proxy/rate limiting
- **ha_get_logbook**: Detect 500 errors, suggest reducing hours_back or adding entity_id filter
- Both tools now reference ha_bug_report for detailed debugging

### Automation Configuration
- **tools_config_automations.py**: Make _normalize_automation_config() recursive
- Now handles nested structures (choose/repeat/if-then-else)
- Normalizes plural→singular fields at all nesting levels

### Tool Documentation
- **ha_get_bulk_status**: Improve description to clarify it's for operation IDs, not entity states
- Add examples and comparison with ha_get_state

## Related Issues
- Closes issues identified in debug report (/github/perso/ha-mcp/debug/)
- Opens #376 (ha_get_notifications)
- Opens #377 (ha_get_states bulk getter)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: move websocket import outside retry loop

Address Gemini Code Assist review suggestion to improve efficiency
by moving import statement outside the retry loop.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Author: julienld

src/ha_mcp/client/rest_client.py

@@ -586,35 +586,59 @@ async def delete_automation_config(self, identifier: str) -> dict[str, Any]:
             raise
 
     async def send_websocket_message(self, message: dict[str, Any]) -> dict[str, Any]:
-        """Send message via WebSocket and wait for response."""
-        ws_client = None
-        try:
-            # Use client's own URL and token for WebSocket connection
-            from .websocket_client import HomeAssistantWebSocketClient
+        """Send message via WebSocket and wait for response.
+
+        Uses the global WebSocket singleton to avoid race conditions from
+        parallel tool calls creating multiple simultaneous connections.
+        """
+        from .websocket_client import get_websocket_client
 
-            ws_client = HomeAssistantWebSocketClient(self.base_url, self.token)
+        max_retries = 2
+        retry_delay = 0.5  # seconds
 
-            # Connect if not already connected
-            if not ws_client.is_connected:
-                await ws_client.connect()
+        for attempt in range(max_retries):
+            try:
+                # Use singleton WebSocket client (shared, reused connection)
+                ws_client = await get_websocket_client()
 
-            # Special handling for render_template which returns an event with the actual result
-            if message.get("type") == "render_template":
-                return await self._handle_render_template(ws_client, message)
+                # Special handling for render_template which returns an event with the actual result
+                if message.get("type") == "render_template":
+                    return await self._handle_render_template(ws_client, message)
 
-            # Extract command type and parameters for other commands
-            message_copy = message.copy()
-            command_type = message_copy.pop("type")
-            result = await ws_client.send_command(command_type, **message_copy)
+                # Extract command type and parameters for other commands
+                message_copy = message.copy()
+                command_type = message_copy.pop("type")
+                result = await ws_client.send_command(command_type, **message_copy)
 
-            return result
-        except Exception as e:
-            logger.error(f"WebSocket message failed: {e}")
-            return {"success": False, "error": str(e)}
-        finally:
-            # Clean up WebSocket connection
-            if ws_client and ws_client.is_connected:
-                await ws_client.disconnect()
+                return result
+
+            except Exception as e:
+                error_str = str(e)
+
+                # Detect transient 403 errors (rate limiting / reverse proxy throttling)
+                if "403" in error_str and "Forbidden" in error_str:
+                    if attempt < max_retries - 1:
+                        logger.warning(
+                            f"WebSocket 403 error (attempt {attempt + 1}/{max_retries}), "
+                            f"retrying after {retry_delay}s: {error_str}"
+                        )
+                        await asyncio.sleep(retry_delay)
+                        continue
+                    else:
+                        logger.error(f"WebSocket 403 error after {max_retries} attempts: {error_str}")
+                        return {
+                            "success": False,
+                            "error": f"WebSocket request blocked (403 Forbidden): {error_str}",
+                            "suggestions": [
+                                "This may be caused by a reverse proxy or security filter",
+                                "Try simplifying the request (e.g., shorter templates, fewer parameters)",
+                                "If using complex templates, try breaking them into smaller parts",
+                                "Check if your Home Assistant is behind a reverse proxy with security rules",
+                            ],
+                        }
+
+                logger.error(f"WebSocket message failed: {e}")
+                return {"success": False, "error": str(e)}
 
     async def _handle_render_template(
         self, ws_client: Any, message: dict[str, Any]

src/ha_mcp/tools/tools_config_automations.py

@@ -21,37 +21,54 @@
 logger = logging.getLogger(__name__)
 
 
-def _normalize_automation_config(config: dict[str, Any]) -> dict[str, Any]:
+def _normalize_automation_config(config: Any) -> Any:
     """
-    Normalize automation config field names to HA API format.
+    Recursively normalize automation config field names to HA API format.
 
     Home Assistant accepts both singular ('trigger', 'action', 'condition')
     and plural ('triggers', 'actions', 'conditions') field names in YAML,
     but the API expects singular forms. This function normalizes plural
-    to singular for consistency.
+    to singular for consistency, recursively processing nested structures
+    like 'choose', 'repeat', 'if/then/else'.
 
     Args:
-        config: Automation configuration dict
+        config: Automation configuration (dict, list, or primitive)
 
     Returns:
         Normalized configuration with singular field names
     """
+    # Handle lists - recursively process each item
+    if isinstance(config, list):
+        return [_normalize_automation_config(item) for item in config]
+
+    # Handle primitives (strings, numbers, etc.)
+    if not isinstance(config, dict):
+        return config
+
+    # Process dictionary
     normalized = config.copy()
 
     # Map plural field names to singular (HA API format)
     field_mappings = {
         "triggers": "trigger",
         "actions": "action",
         "conditions": "condition",
+        # Note: 'sequence' is already singular, but some users might use 'sequences'
+        "sequences": "sequence",
     }
 
+    # Apply field mapping to current level
     for plural, singular in field_mappings.items():
         if plural in normalized and singular not in normalized:
             normalized[singular] = normalized.pop(plural)
         elif plural in normalized and singular in normalized:
             # Both exist - prefer singular, remove plural
             del normalized[plural]
 
+    # Recursively process all values in the dictionary
+    for key, value in normalized.items():
+        normalized[key] = _normalize_automation_config(value)
+
     return normalized
 
 

src/ha_mcp/tools/tools_service.py

@@ -258,9 +258,35 @@ async def ha_bulk_control(
         )
         return cast(dict[str, Any], result)
 
-    @mcp.tool(annotations={"readOnlyHint": True, "title": "Get Bulk Status"})
+    @mcp.tool(annotations={"readOnlyHint": True, "title": "Get Bulk Operation Status"})
     async def ha_get_bulk_status(operation_ids: list[str]) -> dict[str, Any]:
-        """Check status of multiple WebSocket-monitored operations."""
+        """
+        Check status of multiple device control operations.
+
+        Use this tool to check the status of operations initiated by ha_bulk_control
+        or control_device_smart. Each of these tools returns unique operation_ids
+        that can be tracked here.
+
+        **IMPORTANT:** This tool is for tracking async device operations, NOT for
+        checking current entity states. To get current states of entities, use
+        ha_get_state instead.
+
+        **Args:**
+            operation_ids: List of operation IDs returned by ha_bulk_control or
+                          control_device_smart (e.g., ["op_1234", "op_5678"])
+
+        **Returns:**
+            Status summary with completion/pending/failed counts and detailed
+            results for each operation.
+
+        **Example:**
+            # After calling control_device_smart
+            result = control_device_smart("light.kitchen", "on")
+            op_id = result["operation_id"]  # e.g., "op_1234"
+
+            # Check operation status
+            status = ha_get_bulk_status([op_id])
+        """
         result = await device_tools.get_bulk_operation_status(
             operation_ids=operation_ids
         )

src/ha_mcp/tools/tools_utility.py

@@ -189,10 +189,25 @@ async def ha_get_logbook(
             return await add_timezone_metadata(client, logbook_data)
 
         except Exception as e:
+            error_str = str(e)
+            suggestions = []
+
+            # Detect 500 errors (server crash from heavy query)
+            if "500" in error_str:
+                suggestions = [
+                    "The query returned too many results causing a server error (500).",
+                    "This often happens with very active entities or long time periods.",
+                    "Try reducing 'hours_back' parameter (e.g., from 24 to 1 hour)",
+                    "Add a specific 'entity_id' filter to narrow down results",
+                    "If debugging an automation, filter by that automation's entity_id",
+                    "Use ha_bug_report tool to check Home Assistant logs for crash details",
+                ]
+
             error_data = {
                 "success": False,
-                "error": f"Failed to retrieve logbook: {str(e)}",
+                "error": f"Failed to retrieve logbook: {error_str}",
                 "period": f"{hours_back_int} hours back from {end_dt.isoformat()}",
+                "suggestions": suggestions if suggestions else None,
             }
             return await add_timezone_metadata(client, error_data)
 
@@ -389,17 +404,32 @@ async def ha_eval_template(
                 }
 
         except Exception as e:
+            error_str = str(e)
+            suggestions = [
+                "Check Home Assistant WebSocket connection",
+                "Verify template syntax is valid Jinja2",
+                "Try a simpler template to test basic functionality",
+                "Check if referenced entities exist",
+                "Ensure template doesn't exceed timeout limit",
+            ]
+
+            # Add specific suggestions for 403 errors
+            if "403" in error_str and "Forbidden" in error_str:
+                suggestions = [
+                    "The request was blocked (403 Forbidden) - this may be caused by:",
+                    "  • Reverse proxy security rules (Apache, Nginx, Traefik)",
+                    "  • Rate limiting from multiple simultaneous requests",
+                    "  • Complex template triggering security filters",
+                    "Try simplifying the template (remove newlines, reduce complexity)",
+                    "Break complex templates into multiple simpler calls",
+                    "Use ha_bug_report tool to check Home Assistant logs for details",
+                ] + suggestions
+
             return {
                 "success": False,
                 "template": template,
-                "error": f"Template evaluation failed: {str(e)}",
-                "suggestions": [
-                    "Check Home Assistant WebSocket connection",
-                    "Verify template syntax is valid Jinja2",
-                    "Try a simpler template to test basic functionality",
-                    "Check if referenced entities exist",
-                    "Ensure template doesn't exceed timeout limit",
-                ],
+                "error": f"Template evaluation failed: {error_str}",
+                "suggestions": suggestions,
             }
 
     @mcp.tool(annotations={"readOnlyHint": True, "title": "Get Domain Docs"})