feat(tdpilot_API): Tier 1+2 port — official docs + TD 2025 + introspect + memory advanced + validate (66→88 tools)

Adds 22 tools the CLI had exclusively. Standalone now covers ~85% of the
CLI surface. Schema cost: ~33% growth (~4KB) — almost entirely absorbed
by DeepSeek's auto-cache (~0.5% real per-turn cost increase).

New modules:
- tdpilot_api_official_docs.py — 8 tools routing through the existing
  knowledge_search/knowledge_get BM25 with smart filters. Five Tier 1
  lookups (search_official_docs, get_operator_doc, get_param_help,
  lookup_snippets, lookup_palette_component) + three Tier 2 LLM-flavored
  recommenders (recommend_official_component, find_official_example,
  explain_better_way). Without the derivative corpus on disk, returns
  structured hints rather than failing.
- tdpilot_api_td2025.py — 7 read-only TD-runtime probes:
  python_env_status, threading_status, logger_status, tdresources_inspect,
  color_pipeline, component_standardize (per-COMP standards audit),
  audit_project (recursive subtree audit aggregating errors + warnings
  + standards-compliance issues, capped at max_depth).
- tdpilot_api_introspect.py — 3 server-side introspection tools:
  get_server_metrics (uptime + RSS + CPU + thread count + runtime
  counters via optional psutil), describe_surface (categorised tool
  list, both built-in + user-pluggable), get_capabilities (feature
  flags derived from module-import success).

Extensions to existing modules:
- tdpilot_api_memory.py: memory_export (full JSON dump), memory_import
  (with overwrite=False default to preserve local edits), memory_favorite
  (frontmatter flag + 0-5 rating).
- tdpilot_api_recipes.py: td_validate_recipe — pre-save sanity check
  validating replay JSON shape + that every step's tool name is in the
  live TOOL_TO_HANDLER (built-in or user-pluggable).

Wired into:
- tdpilot_api_extension.py — three new modules added to
  _ensure_module_path + handler_modules list
- build_tdpilot_api_tox.py — three new entries in _SOURCE_FILES so the
  rebuilt .tox bakes them as textDATs
- tdpilot_api_schema_defs.py — 22 new TOOL_SCHEMAS entries (input_schemas
  with required fields + descriptions)
- tdpilot_api_schema_map.py — 22 new TOOL_TO_HANDLER mappings

Tests (49 new, 930 total — was 881):
- tests/test_tdpilot_api_official_docs.py — 18 tests with patched
  knowledge_search/knowledge_get to verify the routing layer (filter
  application, fallback paths, hint surfaces) without needing a real
  corpus on disk.
- tests/test_tdpilot_api_td2025.py — 11 tests covering Python env +
  threading probes (no TD needed) and the structured "outside TD" error
  path for TD-specific probes.
- tests/test_tdpilot_api_introspect.py — 6 tests verifying uptime,
  surface description, capability reporting work outside TD.
- tests/test_tdpilot_api_memory.py — 8 new tests for export/import/
  favorite (export round-trip, import skip-vs-overwrite semantics,
  favorite frontmatter writes, rating validation, missing-name error).
- tests/test_tdpilot_api_recipes.py — 6 new tests for td_validate_recipe
  covering the happy path + every issue type (unknown tool, non-dict
  step, missing tool field, non-dict args, non-list replay).

Docs updated:
- README.md: tools badge 66 → 88; comparison table; "what CLI exclusively
  has" trimmed to the durable gap (15 tools); "now in BOTH" callout for
  the ported categories.
- docs/MANUAL.md: tool surface 66→88, capability matrix re-categorized
  (15 rows moved from CLI-only to BOTH; new memory-advanced row notes
  partial coverage), tools table grows 13→16 categories with the new
  ones listed.

Performance: all 22 new tools execute in single-digit milliseconds
(BM25 lookup, sys.* probe, or dict construction). No streaming, no
polling, no cook-thread blocking. handle_audit_project is the heaviest
(O(N) walk over project subtree) and is bounded by max_depth.

NOTE: tdpilot_API.tox needs rebuilding inside TouchDesigner before the
new tools are live. The .tox-source-hash gate only watches the dpsk4
.tox sources, so CI passes — but the standalone .tox loaded in TD won't
have the three new textDATs baked until rebuild. Use the standalone
build snippet from the README.

Author: dreamrec

README.md

@@ -14,7 +14,7 @@
 [![downloads](https://img.shields.io/npm/dm/tdpilot-dpsk4?label=downloads)](https://www.npmjs.com/package/tdpilot-dpsk4)
 [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
 [![python](https://img.shields.io/badge/python-3.10%2B-blue)](./pyproject.toml)
-[![tools](https://img.shields.io/badge/tools-66%20%28standalone%29%20%C2%B7%20103%20%28CLI%29-blueviolet)](./docs/MANUAL.md)
+[![tools](https://img.shields.io/badge/tools-88%20%28standalone%29%20%C2%B7%20103%20%28CLI%29-blueviolet)](./docs/MANUAL.md)
 [![TouchDesigner](https://img.shields.io/badge/TouchDesigner-2025.30000%2B-ff6200)](https://derivative.ca)
 [![DeepSeek](https://img.shields.io/badge/DeepSeek-v4-00a86b)](https://deepseek.com)
 
@@ -26,7 +26,7 @@ There are two ways to run it. Pick whichever fits — they coexist in the same T
 |---|---|---|
 | **Install effort** | Drag one file in, paste a key. | Install Claude Code, install the plugin, configure MCP. |
 | **Where chat lives** | Browser tab + a panel inside TD. | Your Claude Code terminal. |
-| **Tools** | 66 curated for in-TD use | 103 (full surface) |
+| **Tools** | 88 curated for in-TD use | 103 (full surface) |
 | **Best for** | Live performance, quick patches, demos, "no setup" use | Heavy multi-file projects, long sessions, full Claude Code ecosystem |
 | **TD port** | 9987 | 9985 + 9986 |
 | **Config dir** | `~/.tdpilot-api/` | `~/.tdpilot-dpsk4/` |
@@ -168,24 +168,22 @@ Both variants run on the same DeepSeek backend and share the same TD-side handle
 - You want zero CLI dependencies. No Node, no uv, no Claude Code install.
 - You like the chat panel rendering inside TD's UI itself.
 
-**Pick the Claude Code CLI if** you're already using Claude Code, want long sessions across TD + non-TD code, or need any of the 37 tools that only exist on the CLI:
+**Pick the Claude Code CLI if** you're already using Claude Code, want long sessions across TD + non-TD code, or need any of the 15 tools that only exist on the CLI:
 
 | Category | What the CLI exclusively gives you | Tools |
 |---|---|---|
 | **Typed patch sessions** | Plan a multi-step build, dry-run it, apply it, validate, generate variations — wrapped in a typed `PatchPlan` value with atomic rollback on failure. | `td_plan_patch`, `td_preflight_patch`, `td_patch_apply`, `td_patch_validate`, `td_patch_variations` |
 | **Streaming output** | Push live TOP frames over WebSocket to a client at controllable FPS. | `td_stream_top`, `td_stop_stream_top` |
 | **Continuous vision monitoring** | Watch a TOP over time — alpha coverage, luminance, dominant color, ROI diff between frames. | `td_monitor_visual`, `td_capture_and_analyze` |
 | **Visual optimization** | Suggest improvements to a render based on weighted objectives (stability vs complexity). | `td_optimize_visual` |
-| **Official-docs lookup** | Search and quote TouchDesigner's official documentation corpus from inside the agent. | `td_search_official_docs`, `td_get_operator_doc`, `td_get_param_help`, `td_lookup_snippets`, `td_lookup_palette_component` |
-| **Recommendations** | Surface official Palette components and TD-shipped examples instead of building from scratch. | `td_recommend_official_component`, `td_find_official_example`, `td_explain_better_way` |
-| **TD 2025 native introspection** | Inspect Python env, threading, logger, TDResources, color pipeline, COMP-standardize audits. | `td_python_env_status`, `td_threading_status`, `td_logger_status`, `td_tdresources_inspect`, `td_component_standardize`, `td_color_pipeline` |
-| **Advanced memory** | Auto-extract a recipe from a live network, replay it elsewhere, promote project techniques to global, export/import the library. | `td_memory_learn`, `td_memory_replay`, `td_memory_promote`, `td_memory_export`, `td_memory_import`, `td_memory_favorite` |
-| **Macros & planning advanced** | Author macros from templates, audit a whole project subtree, validate technique recipes. | `td_create_macro`, `td_audit_project`, `td_validate_recipe` |
-| **Server introspection** | Health metrics, surface description, capability reporting. | `td_get_server_metrics`, `td_describe_surface`, `td_get_capabilities` |
+| **Advanced memory** | Auto-extract a recipe from a live network, replay it elsewhere, promote project techniques to global. | `td_memory_learn`, `td_memory_replay`, `td_memory_promote` |
+| **Macros & planning advanced** | Author macros from templates programmatically. | `td_create_macro` |
 | **Slash commands** | One-keystroke `/td-check` (project health) and `/td-snapshot` (safety snapshot). | `/td-check`, `/td-snapshot` |
 | **Plugin marketplace + skills** | Auto-activating skills for TD work; full Claude Code skill ecosystem available alongside. | `tdpilot-dpsk4-core`, `tdpilot-dpsk4-production`, `popx-touchdesigner` |
 
-The standalone has 66 tools that cover the everyday inspect → build → wire → verify loop, plus persistent memory, knowledge corpus, recipes, snapshots, subagents (parallel fan-out), multi-model routing (auto/flash/pro), macros, and user-pluggable tools (drop a `.py` in `~/.tdpilot-api/tools/`).
+**Now in BOTH variants** (Tier 1+2 ports landed in the standalone): official-docs lookup (5 tools), TD 2025 native introspection (6 tools), recommendations (3 tools), server introspection (3 tools), audit/validate utilities (2 tools), memory export/import/favorite (3 tools).
+
+The standalone has 88 tools that cover the everyday inspect → build → wire → verify loop, plus persistent memory, knowledge corpus, recipes, snapshots, subagents (parallel fan-out), multi-model routing (auto/flash/pro), macros, user-pluggable tools (drop a `.py` in `~/.tdpilot-api/tools/`), official-docs lookup against the derivative corpus, TD 2025 runtime introspection (Python env, threading, color pipeline), and project-audit + recipe-validation utilities.
 
 **Run both at the same time.** The two .tox files coexist in the same TD project — different ports, different config dirs, different COMP names. Standalone in the browser for quick chat, CLI in the terminal for heavy work.
 

docs/MANUAL.md

@@ -116,7 +116,7 @@ Both run on the same DeepSeek backend and share the TD-side handler layer. The c
 | **Where you type** | Browser tab or in-TD panel | Your terminal |
 | **Setup time** | ~2 min (drag + paste key) | ~10 min (Claude Code + plugin + bridge .tox) |
 | **Dependencies** | TouchDesigner + a browser | TouchDesigner + Node.js + uv + Claude Code |
-| **Tool surface** | 66 tools | 103 tools |
+| **Tool surface** | 88 tools | 103 tools |
 | **Multi-file projects** | TD-only | Anything Claude Code touches (TD + Python + Markdown + …) |
 | **Session length** | Lighter — one focused conversation per turn | Heavier — long sessions across many files |
 | **Collaboration** | Multi-tab — open the chat URL on a phone or second monitor | Terminal stays with one user |
@@ -143,18 +143,21 @@ Both run on the same DeepSeek backend and share the TD-side handler layer. The c
 | Multi-model routing (auto/flash/pro) | ✅ | — | Standalone only |
 | Macro engine (5 bundled templates) | ✅ | — | Standalone only |
 | User-pluggable tools (`~/.tdpilot-api/tools/*.py`) | ✅ | — | Standalone only |
+| **Official-docs lookup** (`td_search_official_docs`, `td_get_operator_doc`, `td_get_param_help`, `td_lookup_snippets`, `td_lookup_palette_component`) | ✅ | ✅ | Both — searches the auto-discovered `derivative` corpus |
+| **TD 2025 native introspection** (`td_python_env_status`, `td_threading_status`, `td_logger_status`, `td_tdresources_inspect`, `td_component_standardize`, `td_color_pipeline`) | ✅ | ✅ | Both — read-only Python/TD probes |
+| **Project audit** (`td_audit_project`) | ✅ | ✅ | Both — recursive subtree audit |
+| **Recipe validation** (`td_validate_recipe`) | ✅ | ✅ | Both — pre-save sanity check |
+| **Recommendations** (`td_recommend_official_component`, `td_find_official_example`, `td_explain_better_way`) | ✅ | ✅ | Both — LLM-flavored docs surfacing |
+| **Server introspection** (`td_get_server_metrics`, `td_describe_surface`, `td_get_capabilities`) | ✅ | ✅ | Both — runtime metrics + tool surface description |
+| **Memory advanced — export/import/favorite** (`memory_export`, `memory_import`, `memory_favorite`) | ✅ | partial | Standalone has all three; CLI has the analogous `td_memory_export`/`import`/`favorite` plus `td_memory_learn`/`replay`/`promote` |
+| **Memory advanced — learn/replay/promote** (`td_memory_learn`, `td_memory_replay`, `td_memory_promote`) | — | ✅ | CLI only — recipe extraction + cross-project promotion |
 | **Typed patch-session API** (`td_plan_patch` → `td_preflight_patch` → `td_patch_apply` → `td_patch_validate` → `td_patch_variations`) | — | ✅ | CLI only — atomic multi-step builds with typed rollback |
-| **Streaming TOP output** (`td_stream_top`, `td_stop_stream_top`) | — | ✅ | CLI only |
-| **Continuous vision monitoring** (`td_monitor_visual`, `td_capture_and_analyze`) | — | ✅ | CLI only |
-| **Visual optimization** (`td_optimize_visual`) | — | ✅ | CLI only |
-| **Server introspection** (`td_get_server_metrics`, `td_describe_surface`, `td_get_capabilities`) | — | ✅ | CLI only |
-| **Official-docs lookup** (`td_search_official_docs`, `td_get_operator_doc`, `td_get_param_help`, `td_lookup_snippets`, `td_lookup_palette_component`) | — | ✅ | CLI only — TD's official docs corpus |
-| **TD 2025 native** (`td_python_env_status`, `td_threading_status`, `td_logger_status`, `td_tdresources_inspect`, `td_component_standardize`, `td_color_pipeline`) | — | ✅ | CLI only |
-| **Recommendations** (`td_recommend_official_component`, `td_find_official_example`, `td_explain_better_way`) | — | ✅ | CLI only |
-| **Memory advanced** (`td_memory_learn`, `td_memory_replay`, `td_memory_promote`, `td_memory_export`/`import`, `td_memory_favorite`) | — | ✅ | CLI only — recipe extraction + cross-project promotion |
-| **Macros & planning advanced** (`td_create_macro`, `td_audit_project`, `td_validate_recipe`) | — | ✅ | CLI only |
+| **Macro authoring** (`td_create_macro`) | — | ✅ | CLI only — programmatic macro construction |
+| **Streaming TOP output** (`td_stream_top`, `td_stop_stream_top`) | — | ✅ | CLI only — interactive workflow doesn't need this |
+| **Continuous vision monitoring** (`td_monitor_visual`, `td_capture_and_analyze`) | — | ✅ | CLI only — for unattended autonomous agents |
+| **Visual optimization** (`td_optimize_visual`) | — | ✅ | CLI only — multi-pass batch workflow |
 | **Slash commands** (`/td-check`, `/td-snapshot`) | — | ✅ | Claude Code plugin |
-| **Tool count** | 66 | 103 | — |
+| **Tool count** | 88 | 103 | — |
 | **Where chat lives** | Browser tab + in-TD panel | Your terminal | — |
 | **Setup time** | ~2 min | ~10 min | — |
 | **Dependencies** | TouchDesigner + browser | TD + Node.js + uv + Claude Code | — |
@@ -256,7 +259,7 @@ The chat lives at `http://127.0.0.1:9987/` and is also rendered inside the COMP'
 
 ## Tools
 
-66 tools across 13 categories. Full schemas in `td_component/tdpilot_api_schema_defs.py`.
+88 tools across 16 categories. Full schemas in `td_component/tdpilot_api_schema_defs.py`.
 
 | Category | Count | Examples |
 |---|---|---|
@@ -273,8 +276,14 @@ The chat lives at `http://127.0.0.1:9987/` and is also rendered inside the COMP'
 | Patch sessions | 6 | `snapshot_save`, `snapshot_list`, `patch_begin`, `patch_validate`, `patch_commit`, `patch_rollback` |
 | Subagents | 5 | `spawn_subagent`, `subagent_status`, `subagent_wait`, `subagent_cancel`, `subagent_list` |
 | Macros & user tools | 5 | `macro_list`, `macro_get`, `macro_run`, `tool_list_user`, `tool_validate` |
-
-The CLI variant adds 37 more tools — see [Standalone vs CLI](#standalone-vs-cli--practical-differences) for the gap.
+| Memory advanced | 3 | `memory_export`, `memory_import`, `memory_favorite` |
+| Recipe validation | 1 | `td_validate_recipe` |
+| Official-docs lookup | 5 | `td_search_official_docs`, `td_get_operator_doc`, `td_get_param_help`, `td_lookup_snippets`, `td_lookup_palette_component` |
+| Recommendations | 3 | `td_recommend_official_component`, `td_find_official_example`, `td_explain_better_way` |
+| TD 2025 native | 7 | `td_python_env_status`, `td_threading_status`, `td_logger_status`, `td_tdresources_inspect`, `td_color_pipeline`, `td_component_standardize`, `td_audit_project` |
+| Server introspection | 3 | `td_get_server_metrics`, `td_describe_surface`, `td_get_capabilities` |
+
+The CLI variant adds 15 more tools — see [Standalone vs CLI](#standalone-vs-cli--practical-differences) for the gap.
 
 ---
 

td_component/build_tdpilot_api_tox.py

@@ -196,6 +196,9 @@ def _load_legacy_module():
     ("tdpilot_api_user_tools", "textDAT", "td_component/tdpilot_api_user_tools.py"),
     ("tdpilot_api_subagents", "textDAT", "td_component/tdpilot_api_subagents.py"),
     ("tdpilot_api_macros", "textDAT", "td_component/tdpilot_api_macros.py"),
+    ("tdpilot_api_official_docs", "textDAT", "td_component/tdpilot_api_official_docs.py"),
+    ("tdpilot_api_td2025", "textDAT", "td_component/tdpilot_api_td2025.py"),
+    ("tdpilot_api_introspect", "textDAT", "td_component/tdpilot_api_introspect.py"),
     ("tdpilot_api_chat_html", "textDAT", "td_component/tdpilot_api_chat.html"),
     ("tdpilot_api_web_callbacks", "textDAT", "td_component/tdpilot_api_web_callbacks.py"),
     ("mcp_webserver_callbacks", "textDAT", "td_component/mcp_webserver_callbacks.py"),

td_component/tdpilot_api_extension.py

@@ -108,6 +108,9 @@ def _ensure_module_path(self) -> None:
             "tdpilot_api_user_tools",
             "tdpilot_api_subagents",
             "tdpilot_api_macros",
+            "tdpilot_api_official_docs",
+            "tdpilot_api_td2025",
+            "tdpilot_api_introspect",
             "mcp_webserver_callbacks",
         ):
             child = self.owner.op(name)
@@ -199,6 +202,9 @@ def _build_runtime(self) -> None:
                 ("tdpilot_api_user_tools", "user_tools"),
                 ("tdpilot_api_subagents", "subagents"),
                 ("tdpilot_api_macros", "macros"),
+                ("tdpilot_api_official_docs", "official_docs"),
+                ("tdpilot_api_td2025", "td2025_native"),
+                ("tdpilot_api_introspect", "introspect"),
             ):
                 dat = self.owner.op(mod_name)
                 if dat is not None: