feat: OpenClaw integration - ClawHub skill, MCP structured output, and E2E tests (#272)

* feat: OpenClaw ClawHub skill and MCP structured output

Add a ClawHub-compatible skill (skills/leann-memory/) that enables
OpenClaw agents to use LEANN for 97% storage-compressed memory search
with free local embeddings. No PR to the OpenClaw repo required —
users install via `clawhub install` or copy the skill directory.

Skill files:
- claw.json: ClawHub manifest with permissions, tags, model compat
- instructions.md: Agent-facing instructions for search workflow
- README.md: Human-facing setup and comparison table

MCP server:
- Add --json flag to leann_search tool for structured output
  (enables machine-readable results for both MCP and skill paths)

Docs:
- openclaw-setup.md: Quick start guide with MCP and skill options
- openclaw-integration-plan.md: Full competitive analysis, storage
  growth calculations, and marketing strategy

Made-with: Cursor

* feat: OpenClaw integration tests, clean JSON output, and type fixes

- Add tests/openclaw/ suite: skill manifest, MCP protocol, build/search
  E2E, and subprocess-based MCP E2E tests
- Fix SearchResult.score to emit native float (not numpy.float32)
- Add --json quiet mode to suppress informational prints to stderr
- Redirect C++ diagnostic printf to stderr in faiss submodule
- Add stdout fd-level redirection with libc.fflush for clean JSON output
- Fix ty type checker errors (assert shutil.which not None, pytest.skip)

Made-with: Cursor

* feat: real OpenClaw E2E integration test

- Add test_openclaw_e2e.py: 11 tests against a live OpenClaw Docker
  instance with real memory formation via qwen3:8b + native Ollama API
- Fix docker-compose.yml: use --bind lan and native ollama API for
  proper tool calling (openai-completions silently breaks it)
- Add tests/openclaw/.gitignore for docker-data/

Made-with: Cursor

* chore: remove internal strategy docs from public repo

Made-with: Cursor

* docs: restore openclaw-setup.md (user-facing guide)

Made-with: Cursor

* chore: upgrade pytest to 9.0 and remove skip workarounds

pytest 9 fixes the type annotation for pytest.skip(), so ty passes
without the `raise pytest.skip.Exception(...)` workaround. Reverted
all skip calls back to the clean `pytest.skip(...)` form.

Made-with: Cursor

* fix: ruff 0.12.7 format and ty type narrowing for CI

- Reformat assert statements to match ruff 0.12.7 style
- Use assert narrowing instead of explicit type annotation for
  leann_mcp_bin after pytest.skip

Made-with: Cursor

Author: andylizf

.gitignore

@@ -21,6 +21,7 @@ demo/experiment_results/**/*.json
 *.png
 !.vscode/*.json
 !.devcontainer/*.json
+!skills/**/claw.json
 *.sh
 *.txt
 !CMakeLists.txt

docs/openclaw-setup.md

@@ -0,0 +1,166 @@
+# OpenClaw + LEANN Setup Guide
+
+Two ways to connect LEANN to your OpenClaw agent: **MCP server** (recommended)
+or **ClawHub skill**.
+
+---
+
+## Option A: MCP Server (Recommended)
+
+OpenClaw natively supports MCP tools. LEANN ships an MCP server that exposes
+`leann_search` and `leann_list` as tools your agent can call directly.
+
+### 1. Install LEANN
+
+```bash
+pip install leann-core
+# or
+uv tool install leann-core --with leann
+```
+
+### 2. Build an index on your memory files
+
+Using Ollama embeddings (recommended if you already run Ollama):
+
+```bash
+leann build openclaw-memory \
+  --docs ~/.openclaw/workspace/MEMORY.md ~/.openclaw/workspace/memory/ \
+  --embedding-mode ollama \
+  --embedding-model nomic-embed-text
+```
+
+Or using local sentence-transformers (no Ollama required):
+
+```bash
+leann build openclaw-memory \
+  --docs ~/.openclaw/workspace/MEMORY.md ~/.openclaw/workspace/memory/ \
+  --embedding-mode sentence-transformers \
+  --embedding-model all-MiniLM-L6-v2
+```
+
+Add extra directories if you have them:
+
+```bash
+leann build openclaw-memory \
+  --docs ~/.openclaw/workspace/MEMORY.md \
+        ~/.openclaw/workspace/memory/ \
+        ~/Documents/notes/ \
+  --embedding-mode ollama \
+  --embedding-model nomic-embed-text
+```
+
+### 3. Register the MCP server with OpenClaw
+
+Add to `~/.openclaw/openclaw.json`:
+
+```json5
+{
+  // ... your existing config ...
+  "mcpServers": {
+    "leann": {
+      "command": "leann_mcp",
+      "args": [],
+      "env": {}
+    }
+  }
+}
+```
+
+### 4. Use it
+
+Ask your agent:
+- "Search my memories for database decisions"
+- "What did we decide about the API design?"
+- "Find my notes on deployment"
+
+The agent will call `leann_search` via MCP and return structured results.
+
+### 5. Keep the index fresh
+
+```bash
+# Re-run build (idempotent — only processes changed files)
+leann build openclaw-memory \
+  --docs ~/.openclaw/workspace/MEMORY.md ~/.openclaw/workspace/memory/
+
+# Or use watch mode for continuous auto-sync
+leann watch openclaw-memory --interval 30
+```
+
+---
+
+## Option B: ClawHub Skill
+
+If you prefer the skill-based approach:
+
+```bash
+clawhub install leann-team/leann-memory
+```
+
+Or copy `skills/leann-memory/` from this repo to
+`~/.openclaw/workspace/skills/leann-memory/`.
+
+The skill tells your agent how to call `leann search` via shell commands.
+Setup steps (install + build index) are the same as above.
+
+---
+
+## Important: Ollama Configuration
+
+If you use Ollama as your OpenClaw model provider, make sure your
+`~/.openclaw/openclaw.json` uses the **native Ollama API** — not the
+OpenAI-compatible endpoint:
+
+```json5
+{
+  "models": {
+    "providers": {
+      "ollama": {
+        "baseUrl": "http://127.0.0.1:11434",  // no /v1 suffix
+        "apiKey": "ollama-local",
+        "api": "ollama"  // NOT "openai-completions" or "openai-responses"
+      }
+    }
+  }
+}
+```
+
+Using `"openai-completions"` or `"openai-responses"` silently breaks tool
+calling — the model outputs tool calls as plain text instead of structured
+`tool_calls`. See [astral-sh/ty#21243](https://github.com/openclaw/openclaw/issues/21243).
+
+---
+
+## Storage Comparison
+
+| Scenario | Default memory-core | LEANN |
+|---|---|---|
+| 1 year daily logs (~12K chunks) | ~23 MB | **~0.7 MB** |
+| + session transcripts (~100K chunks) | ~190 MB | **~6 MB** |
+| + 10 GB indexed documents (~500K chunks) | ~950 MB | **~30 MB** |
+
+All numbers assume 384-dimensional embeddings (all-MiniLM-L6-v2 or
+nomic-embed-text).
+
+---
+
+## Troubleshooting
+
+**"leann: command not found"**
+Ensure LEANN is on your PATH. If installed via `uv tool install`, run
+`uv tool update-shell` and restart your terminal.
+
+**"Index not found"**
+Run `leann list` to see available indexes. Build one first with `leann build`.
+
+**Slow first search**
+The first query loads the embedding model (~90 MB). Subsequent queries reuse the
+warm daemon and are fast (~0.5s). Use `leann warmup openclaw-memory` to
+pre-warm.
+
+**Memory files changed but search results are stale**
+Re-run `leann build openclaw-memory --docs ...` — it detects changes
+automatically and only re-indexes what changed.
+
+**Agent doesn't use LEANN tools**
+Make sure your Ollama model supports tool calling (e.g. `qwen3:8b` or larger).
+Smaller models like `qwen3:4b` may not reliably invoke tools.

packages/leann-backend-hnsw/third_party/faiss

@@ -1328,7 +1328,7 @@ def search(
                     enriched_results.append(
                         SearchResult(
                             id=string_id,
-                            score=dist,
+                            score=float(dist),
                             text=passage_data["text"],
                             metadata=passage_data.get("metadata", {}),
                         )

packages/leann-core/src/leann/api.py

@@ -2426,14 +2426,17 @@ def _resolve_index_path(
         *,
         non_interactive: bool = True,
         purpose: str = "use",
+        quiet: bool = False,
     ) -> Optional[str]:
         """Resolve index path from current project or registered projects."""
+        _print = (lambda *a, **kw: print(*a, file=sys.stderr, **kw)) if quiet else print
+
         if self.index_exists(index_name):
             return self.get_index_path(index_name)
 
         all_matches = self._find_all_matching_indexes(index_name)
         if not all_matches:
-            print(
+            _print(
                 f"Index '{index_name}' not found. Use 'leann build {index_name} --docs <dir> [<dir2> ...]' to create it."
             )
             return None
@@ -2452,7 +2455,7 @@ def _match_to_path(match: dict[str, Any]) -> str:
                 if match["is_current"]
                 else f"project '{match['project_path'].name}'"
             )
-            print(f"Using index '{index_name}' from {project_info}")
+            _print(f"Using index '{index_name}' from {project_info}")
             return _match_to_path(match)
 
         if non_interactive:
@@ -2463,7 +2466,7 @@ def _match_to_path(match: dict[str, Any]) -> str:
                 if match["is_current"]
                 else f"project '{match['project_path'].name}'"
             )
-            print(
+            _print(
                 f"Found {len(all_matches)} indexes named '{index_name}', using index from {location_desc}"
             )
             return _match_to_path(match)
@@ -2499,11 +2502,13 @@ def _match_to_path(match: dict[str, Any]) -> str:
     async def search_documents(self, args):
         index_name = args.index_name
         query = args.query
+        json_mode = getattr(args, "json", False)
 
         index_path = self._resolve_index_path(
             index_name,
             non_interactive=args.non_interactive,
             purpose="search",
+            quiet=json_mode,
         )
         if not index_path:
             return
@@ -2513,24 +2518,40 @@ async def search_documents(self, args):
         if args.embedding_prompt_template:
             provider_options["prompt_template"] = args.embedding_prompt_template
 
-        searcher = LeannSearcher(
-            index_path=index_path,
-            enable_warmup=args.enable_warmup,
-            use_daemon=args.use_daemon,
-            daemon_ttl_seconds=args.daemon_ttl,
-        )
-        results = searcher.search(
-            query,
-            top_k=args.top_k,
-            complexity=args.complexity,
-            beam_width=args.beam_width,
-            prune_ratio=args.prune_ratio,
-            recompute_embeddings=args.recompute_embeddings,
-            pruning_strategy=args.pruning_strategy,
-            provider_options=provider_options if provider_options else None,
-        )
+        if json_mode:
+            sys.stdout.flush()
+            saved_fd = os.dup(1)
+            devnull = os.open(os.devnull, os.O_WRONLY)
+            os.dup2(devnull, 1)
+            os.close(devnull)
+
+        try:
+            searcher = LeannSearcher(
+                index_path=index_path,
+                enable_warmup=args.enable_warmup,
+                use_daemon=args.use_daemon,
+                daemon_ttl_seconds=args.daemon_ttl,
+            )
+            results = searcher.search(
+                query,
+                top_k=args.top_k,
+                complexity=args.complexity,
+                beam_width=args.beam_width,
+                prune_ratio=args.prune_ratio,
+                recompute_embeddings=args.recompute_embeddings,
+                pruning_strategy=args.pruning_strategy,
+                provider_options=provider_options if provider_options else None,
+            )
+        finally:
+            if json_mode:
+                import ctypes
+
+                libc = ctypes.CDLL(None)
+                libc.fflush(None)
+                os.dup2(saved_fd, 1)
+                os.close(saved_fd)
 
-        if getattr(args, "json", False):
+        if json_mode:
             json_results = [
                 {
                     "id": r.id,

packages/leann-core/src/leann/cli.py

@@ -99,7 +99,6 @@ def handle_request(request):
                         },
                     }
 
-                # Build simplified command with non-interactive flag for MCP compatibility
                 cmd = [
                     "leann",
                     "search",
@@ -108,6 +107,7 @@ def handle_request(request):
                     f"--top-k={args.get('top_k', 5)}",
                     f"--complexity={args.get('complexity', 32)}",
                     "--non-interactive",
+                    "--json",
                 ]
                 if args.get("show_metadata", False):
                     cmd.append("--show-metadata")

packages/leann-core/src/leann/mcp.py

@@ -99,7 +99,7 @@ lint = [
 
 # Test toolchain (no heavy project runtime deps)
 test = [
-    "pytest>=7.0",
+    "pytest>=9.0",
     "pytest-cov>=4.0",
     "pytest-xdist>=3.0",
     "pytest-timeout>=2.0",

pyproject.toml

@@ -0,0 +1,65 @@
+# LEANN Memory Search for OpenClaw
+
+97% storage-compressed semantic memory search with free local embeddings.
+
+## Why
+
+Every OpenClaw memory solution stores full embedding vectors. On a 256 GB Mac
+Mini, heavy users accumulate 500 MB - 6 GB+ of embedding indexes over time.
+LEANN compresses this to ~2% of the original size through graph-based selective
+recomputation, while using high-quality local embeddings (zero API cost).
+
+| Feature | Default memory | LEANN |
+|---|---|---|
+| Storage (50K chunks) | ~75 MB | **~2 MB** |
+| Embedding cost | Remote API ($) | **$0 (local)** |
+| Scale | ~100K chunks | **60M+ passages** |
+
+## Install
+
+```bash
+# Install LEANN
+pip install leann-core
+
+# Install the skill
+clawhub install leann-team/leann-memory
+
+# Or manually: copy this directory to ~/.openclaw/workspace/skills/leann-memory/
+```
+
+## Quick Start
+
+```bash
+# Build index on your memory files
+leann build openclaw-memory \
+  --docs ~/.openclaw/workspace/MEMORY.md ~/.openclaw/workspace/memory/ \
+  --embedding-model all-MiniLM-L6-v2
+
+# Test a search
+leann search openclaw-memory "what did we decide about the database" --json
+```
+
+Then ask your OpenClaw agent: "search my memories for database decisions"
+
+## Auto-Sync
+
+Keep the index updated as new memories are added:
+
+```bash
+# One-shot check and rebuild
+leann watch openclaw-memory --once
+
+# Continuous monitoring (runs in background)
+leann watch openclaw-memory --interval 30
+```
+
+## How It Works
+
+LEANN stores a pruned neighbor graph instead of full embedding vectors. During
+search, embeddings are recomputed on-demand via a local daemon. OpenClaw's async
+"sleep time compute" model makes recomputation latency invisible to users.
+
+## Links
+
+- [LEANN Repository](https://github.com/yichuan-w/LEANN)
+- [Integration Plan](../../docs/openclaw-integration-plan.md)