Theme: Orchestration + distribution maturity. Flow FSM (marquee), full self-update, MCP Config interop, Activity Viz dashboard (stretch).
Authored: 2026-05-18 (alongside v2.5 + v2.6 plans).
Status: not_started. Update this front-matter as phases complete.
Prerequisite: v2.6 SHIPPED. Skill packs full-split happens in v2.7.
For the receiving agent: Cold-start executable plan for v2.7 of TDPilot DPSK4. Largest release in the v2.5-v2.7 arc (Flow FSM is ~1500 LOC alone). Same conventions as
v2.5_IMPLEMENTATION_PLAN.md. Pick phases by Status field; sequence within Flow FSM matters (core → templates → UI).
- v2.6.0 shipped with hybrid retrieval + skill packs (opt-in mode) + web ingestion all stable in production.
- Activity log + journal hints (v2.5.1) load-bearing for Activity Viz dashboard (v2.7.4).
Three structurally distinct gaps:
- Orchestration (
v2.7.1): the canonical kaleidoscope-class wandering build motivated v2.4's 10 bug fixes. Activity log + journal hints (v2.5) help but don't STRUCTURE the build. Flow FSM forces explicit state transitions with audit trail — agent commits to a plan, can backtrack on failure, exports trace as Mermaid for post-hoc analysis. - Distribution maturity (
v2.7.2): v2.5.7 ships read-onlytd_check_for_updates. v2.7.2 closes the loop with full apply-flow (server wheel swap,.toxswap with source-hash check, snapshot of pre-update state). - Interop (
v2.7.3): MCP Config lets our agent consume OTHER MCP servers (Blender, ComfyUI, Resolume). Strategic — opens cross-app pipelines.
(v2.7.4 Activity Viz dashboard is stretch — drop first if schedule slips.)
Same as v2.5/v2.6.
| Phase ID | Item | Effort | Tox rebuild | Status |
|---|---|---|---|---|
v2.7.1-flow-fsm |
Flow Controller FSM core + 4 templates + Mermaid chat UI | 2-3 weeks | API only | not_started |
v2.7.2-self-update |
Full td_self_update + Tox Updater (source-hash gated) |
1 week | MCP + API | not_started |
v2.7.3-mcp-config |
Multi-MCP-server consumer + tiered permissions | 1-1.5 weeks | MCP only | not_started |
v2.7.4-activity-viz |
(Stretch) Telemetry dashboard tab in chat UI | 1 week | API only | not_started |
v2.7.5-skill-split |
Full system-prompt split (move DOMAIN content into skills, evict from SYSTEM_PROMPT_BASE) | 3 days | API only | not_started |
Total scope: 5 phases, ~6 weeks of work.
Sequencing constraint:
v2.7.1is the largest single feature — start first, ship as PR #1 of v2.7.v2.7.2andv2.7.3are independent of each other and of Flow FSM — can run in parallel agents after v2.7.1 stabilizes.v2.7.5(skill-split) is a tidy-up that pairs withv2.7.1's skill-aware system prompts.v2.7.4is stretch — schedule last.
Status: not_started
Branch suggestion: claude/v2.7.1-flow-fsm
Tox rebuild: API only
Marquee feature of v2.7. ~1500-1800 LOC + chat UI work.
The canonical failing prompt "Build a kaleidoscope feedback loop" (v2.4 live-debug source) wandered for 30+ turns before the agent self-diagnosed the numerical bug. Activity log + journal hints (v2.5) help the agent SEE its loops. Flow FSM goes further: it STRUCTURES the build. The agent commits to a plan, declares state transitions, backtracks explicitly on failure, exports the trace as Mermaid.
This is the "co-pilot embedded in the network" feature described in the original v2.2→v3.0 ROADMAP.md north-star vision.
git tag --list "v2.6.0*" # v2.6 shipped
grep -c "_TASK_DONE_RE" td_component/tdpilot_api_runtime.py # confirm sticky-pro pattern exists (reused for flow-end)
ls td_component/tdpilot_api_chat.html # chat HTML exists; Flow panel attaches heresrc/td_mcp/flow/__init__.pysrc/td_mcp/flow/state_machine.py— FSM core:Flow,State,Transitiondataclasses;FlowRunnerorchestratorsrc/td_mcp/flow/mermaid.py— renderer:Flow → mermaid stringfor chat UI panel + exportsrc/td_mcp/flow/templates/build_feedback_loop.yaml— the kaleidoscope classsrc/td_mcp/flow/templates/debug_glsl_shader.yamlsrc/td_mcp/flow/templates/wire_audio_reactive.yamlsrc/td_mcp/flow/templates/inspect_and_optimize.yamlsrc/td_mcp/registry/flow_tools.py— register 6 new MCP toolstests/test_v27_flow.py— ~30 tests (FSM, templates, transitions, Mermaid, persistence)
td_component/tdpilot_api_runtime.py— flow-state-aware system prompt injection: when flow active, prepend current state'sgoalto the turn's system prompttd_component/tdpilot_api_runtime.py— auto-transition hooks on tool results (e.g., flowbuild_feedback_looptransitionssurvey → buildwhen agent callstd_create_node)td_component/tdpilot_api_chat.html— Flow panel: Mermaid.js (CDN, lazy-load) renders current flow with active state highlightedtd_component/tdpilot_api_web_callbacks.py— WS eventflow_state_changefor chat UItd_component/build_tdpilot_api_tox.py::_API_TOX_SOURCE_FILES— include flow templates path if dir-loadedCHANGELOG.md
name: build_feedback_loop
description: Construct a feedback chain with decay tuning
version: 1
initial_state: survey
states:
survey:
goal: Inventory existing nodes; identify free space for chain
suggested_tools: [td_get_info, td_get_nodes, td_locations]
exit_when:
kind: tool_called
tool: td_create_node
transitions_to: build
build:
goal: Create render + feedback TOPs, wire feedback chain
suggested_tools: [td_create_node, td_connect_nodes, td_set_params]
exit_when:
kind: condition
condition: feedback_chain_wired # checked via td_get_connections post-tool
transitions_to: tune
tune:
goal: Adjust decay multipliers to achieve visible accumulation
suggested_tools: [td_set_params, td_capture_frame]
exit_when:
kind: tool_called
tool: td_capture_frame
result_filter: visual_ok # heuristic: non-black, non-saturated
transitions_to: snapshot
snapshot:
goal: Snapshot the working state
suggested_tools: [snapshot_save_scoped]
exit_when:
kind: tool_called
tool: snapshot_save_scoped
transitions_to: complete
complete:
terminal: true
on_abort:
rollback_to: snapshot # restore last snapshot if user aborts@mcp.tool()
async def td_flow_start(template: str, params: dict = {}) -> dict:
"""Start a flow from a named template. Returns flow_id + initial state."""
@mcp.tool()
async def td_flow_status() -> dict:
"""Current flow state, history, suggested next tools."""
@mcp.tool()
async def td_flow_transition(to_state: str, reason: str) -> dict:
"""Explicit transition (agent-initiated). Most transitions are auto-triggered by tool results."""
@mcp.tool()
async def td_flow_abort(reason: str) -> dict:
"""Abort current flow. Runs on_abort.rollback_to if defined."""
@mcp.tool()
async def td_flow_export_mermaid(include_history: bool = True) -> str:
"""Render current flow as Mermaid stateDiagram-v2."""
@mcp.tool()
async def td_flow_list_templates() -> list:
"""List available flow templates with descriptions."""On user prompt, runtime checks template description + first-state goal against prompt using hybrid retrieval (v2.6.1). If RRF score > threshold (e.g., 0.05), suggest flow start in chat UI (banner: "This looks like a 'build feedback loop' task. Start flow? [yes / no / never for this session]").
User can also explicitly invoke via /flow start build_feedback_loop slash command in chat (parsed by HTML chat panel, dispatched as td_flow_start).
Flow state stored in comp.storage['tdpilot_api_flow_state'] so it survives .tox reloads (Phase 1.2.2 auto-mirror pattern). Resume on chat-pipe restart.
- Chat HTML grows a "Flow" tab (alongside existing Chat tab)
- Lazy-loads
mermaid.min.jsfrom CDN (10.6+) when tab opens - Renders
td_flow_export_mermaid()output - Highlights current state via Mermaid's class-based styling
- Auto-refreshes on
flow_state_changeWS event
td_flow_abortalways works — agent can exit flow on irrecoverable error- User can click "Abort flow" in chat banner
- Aborts trigger
on_abort.rollback_toif defined (restores named snapshot)
- FSM core: state transitions valid/invalid, auto-trigger from tool results (8 tests)
- Template loading + schema validation (5 tests)
- Mermaid rendering — pinned strings for each of 4 templates (4 tests)
- Persistence across
.toxreload (usingcomp.storagemock) (3 tests) - Auto-start detection: prompt → template-match via hybrid retrieval (5 tests)
- Abort + rollback (2 tests)
- Tool registration discoverable (1 test)
- End-to-end: full
build_feedback_looptraversal with mock tool dispatches (2 tests)
uv run pytest tests/test_v27_flow.py -v
uv run pytest tests/ --ignore=tests/agent_evals -x -q # ~2150 pass
# Manual: invoke /flow start build_feedback_loop in chat, watch Mermaid panel update through each state
# Manual: re-run kaleidoscope task end-to-end — should now complete in fewer turns with explicit state trace- Risk: FSM rigidity frustrates users. Mitigation:
td_flow_abortalways works; flows are opt-in (default off, must be invoked). - Risk: Auto-start false-positives. Mitigation: RRF threshold tuned conservatively; "never for this session" option.
- Risk: Mermaid CDN unavailable (air-gapped users). Mitigation: vendor Mermaid 10.6 as static asset in
td_component/static/. - Risk: Templates rot when MCP tool surface changes. Mitigation: schema-versioned templates + CI test that every template's suggested_tools exist in tool registry.
After this PR merges (largest single PR of v2.7), mark v2.7.1-flow-fsm as completed. Next: v2.7.2 or v2.7.3 (independent — parallel).
Status: not_started
Branch suggestion: claude/v2.7.2-self-update
Tox rebuild: Both .tox files (UI banner lives in API tox, version-bind in MCP tox)
Depends on: v2.5.7 td_check_for_updates (shipped in v2.5).
v2.5.7 surfaces "update available" but offers no apply-path. v2.7.2 closes the loop: server wheel auto-swap, .tox source-hash-gated swap, snapshot of pre-update state, clear messaging on rebuild-required paths.
Architecturally fraught (per v2.5 audit): .tox files are TD-baked binaries; server can self-update wheels but cannot rebuild .tox outside TD. The mitigation is never auto-apply without user click + source-hash check.
git tag --list "v2.5.7*" # td_check_for_updates exists (or rolled into v2.5.0)
grep "td_check_for_updates" src/td_mcp/registry/ # confirmed presentsrc/td_mcp/lifecycle/self_update.py— orchestrator: download wheel, swap-atomic, snapshot pre-statesrc/td_mcp/lifecycle/tox_updater.py— source-hash compare + atomic.toxswap (or guide to TD-rebuild)tests/test_v27_self_update.py— ~15 tests (mock GitHub API + filesystem)
src/td_mcp/registry/lifecycle_tools.py— registertd_self_update(existing file from v2.5.7)td_component/tdpilot_api_runtime.py— yellow "Update available" banner on chat-pipe start whencheck_for_updatesreportshas_update=Truetd_component/tdpilot_api_chat.html— banner UI with "Preview" / "Apply" / "Dismiss"td_component/tdpilot_api_web_callbacks.py— POST/self-update/applyendpoint (requires re-auth even inAuthmode=open— destructive op)
@mcp.tool()
async def td_self_update(
component: Literal["server", "tox", "all"] = "all",
dry_run: bool = False,
confirm_token: Optional[str] = None, # user-provided; prevents agent-driven auto-apply
) -> dict:
"""Apply available updates. server=swap Python wheel. tox=swap .tox if source-hash matches.
Requires confirm_token (echoed from check_for_updates output) — prevents drive-by application."""td_check_for_updates()returns latest tag + URL + aconfirm_token(HMAC-signed by server-side secret, valid 5min)- Agent presents to user: "Update available: v2.6.2 → v2.7.0. Apply? Token: ..."
- User clicks "Apply" in banner → frontend calls
/self-update/applywith token - Server validates token, downloads new wheel from GitHub Release asset
- Atomic install:
pip install --upgrade <wheel>(ornpm i -g tdpilot-dpsk4@<version>for npm install path) - Surface "restart MCP server" advice (we can't restart ourselves cleanly)
- Compute current
.tox-source-hash.json - Download newer release's source-hash from Release assets
- If source hashes match (no editor-rebuild needed because no source file changed between releases — patch release) → swap
.toxbinaries atomically:- Snapshot current
.toxfiles as.tox.pre-update-<timestamp> - Replace with new
.tox - Trigger
chat_web.par.autorestartpulse + .reloadsrc(per v2.3.0 Bug 11 pattern)
- Snapshot current
- If source hashes differ (minor/major release with source changes) → message:
"Source rebuild required. New release added/changed source files. Open TouchDesigner, run the canonical rebuild snippet from
feedback_td_tox_rebuild_recipe, then re-invoketd_self_update."
Before ANY swap (server wheel OR .tox):
- Snapshot current
.tox→.tox.pre-update-<timestamp>(real copy, not symlink — per v2.2.1 auto-mirror lesson) - Snapshot current
pyproject.toml+src/td_mcp/__init__.pyto a state file - On apply failure, restore from snapshot atomically + log
- Mock GitHub API: latest > current, equal, older (3 tests)
- Source-hash match → atomic swap (3 tests)
- Source-hash mismatch → user-guidance message (1 test)
- Snapshot creation + rollback on failure (3 tests)
- confirm_token HMAC validation (2 tests)
- Tool registered (1 test)
- Banner UI render (mocked WS) (2 tests)
uv run pytest tests/test_v27_self_update.py -v
# Manual (CAREFUL): on a test machine, run td_self_update(dry_run=True), verify advice
# Manual: NEVER test apply path on the main dev machine without backups- Risk: Server-version +
.toxdrift after partial update. Mitigation: version-binding check on every chat-pipe start; "incompatible .tox" banner if mismatch. - Risk: Agent-driven auto-apply (Bug 8 class). Mitigation:
confirm_tokenrequired + Tool Approval gate (v2.5.3) routes through user click. - Risk: Download interrupted mid-swap. Mitigation: download to
<file>.partial, atomic rename only on completion.
Mark completed. v2.7.2 is high-value but high-risk — verify with extra care.
Status: not_started
Branch suggestion: claude/v2.7.3-mcp-config
Tox rebuild: MCP only
Our agent runs OTHER MCP servers' tools today only when the user has wired them into Claude Code / Claude Desktop. Direct integration would let tdpilot-dpsk4 agent (in chat-pipe) consume Blender MCP, ComfyUI MCP, Resolume MCP, etc. — opening cross-app creative pipelines.
The plan: ship a tiered "MCP Config" server-permission system. This phase brings the same to TDPilot.
which mcp # MCP CLI installed
python -c "import mcp; print(mcp.__version__)" # client library available~/.tdpilot-dpsk4/api/mcp_config.json:
{
"servers": {
"blender": {
"transport": "stdio",
"command": ["npx", "-y", "blender-mcp"],
"tier": "read-write",
"tools_allowlist": ["mcp__blender__get_scene_info", "mcp__blender__execute_blender_code"],
"enabled": true
},
"comfyui": {
"transport": "http",
"url": "http://localhost:8188",
"tier": "read-only",
"tools_allowlist": ["mcp__comfyui__queue_prompt", "mcp__comfyui__get_history"],
"enabled": false
}
}
}src/td_mcp/mcp_consumer/__init__.pysrc/td_mcp/mcp_consumer/client.py— wrapsmcpclient library (stdio + http transports)src/td_mcp/mcp_consumer/config.py— load + validate + writemcp_config.jsonsrc/td_mcp/registry/mcp_consumer_tools.py— register 3 new toolstests/test_v27_mcp_consumer.py— ~12 tests
@mcp.tool()
async def td_mcp_list_servers() -> list:
"""List connected MCP servers + their tiers + enabled state."""
@mcp.tool()
async def td_mcp_invoke(server: str, tool: str, args: dict) -> dict:
"""Invoke a tool on a connected MCP server. Respects tier + allowlist."""
@mcp.tool()
async def td_mcp_add_server(name: str, transport: str, command_or_url: str, tier: str) -> dict:
"""Register a new MCP server in config. Requires Tool Approval (v2.5.3)."""read-only: agent can invoke tools whose name suggests read-only (heuristic: starts withget_,list_,info,inspect,query,search). Other tools require Tool Approval per-call.read-write: agent can invoke any tool intools_allowlist. Tools outside allowlist require Tool Approval per-call.full: any tool. Discouraged. Requires explicit user setup.
External MCP tool calls flow through the same approval gate. Two layers:
- Tier + allowlist check (this phase)
- Tool Approval click-through if destructive heuristic fires (v2.5.3)
- Config load/validate roundtrip (2 tests)
- Server add/remove (2 tests)
- Tier-allowlist enforcement (3 tests)
- Read-only heuristic for tool names (2 tests)
- stdio client smoke (mock) (1 test)
- http client smoke (mock) (1 test)
- Tool registration discoverable (1 test)
uv run pytest tests/test_v27_mcp_consumer.py -v
# Manual: add blender MCP via td_mcp_add_server, invoke a read tool, verify result- Risk: External MCP server crashes mid-turn. Mitigation: circuit breaker (v2.4 Phase C.8) extends to external servers.
- Risk: Tier-allowlist bypass via tool-name spoofing. Mitigation: allowlist is exact-match by tool name; no glob.
- Risk: Config file corruption. Mitigation: atomic write; backup file
mcp_config.json.bak.
Mark completed. Move to v2.7.4 or v2.7.5.
Status: not_started
Branch suggestion: claude/v2.7.4-activity-viz
Tox rebuild: API only
Drop first if v2.7 schedule slips.
Visual surface for activity_log + journal_hints + cost-pill + Flow state. All v2.5-v2.7 observability data unified in one panel. Rough analogue to comparable "Activity Viz" panels in third-party references.
td_component/tdpilot_api_chat_telemetry.html— extracted telemetry panel HTML (loaded into Chat tab as iframe or directly imported)td_component/tdpilot_api_telemetry.py— server-side aggregator + WS event emittertests/test_v27_activity_viz.py— ~8 tests (mocked WS, render snapshot)
td_component/tdpilot_api_chat.html— Telemetry tabtd_component/tdpilot_api_web_callbacks.py—/telemetryendpoint streams aggregated data
- Tool-call timeline (horizontal bar, last 50 calls) — sourced from
td_get_activity_log(v2.5.1) - Cost gauge — today / this week / lifetime — sourced from v2.4 cost-tracking (Phase C.7)
- Cycle-detect heatmap —
(tool, args_hash)pairs that hit threshold this session - Active flow + state — current FSM (v2.7.1)
- Active skills — loaded skill packs (v2.6.2)
- Token consumption breakdown — input / cached / output per turn (last 20 turns)
- Tool-result kind distribution — ok/error/no_change ratios from activity log
- Recent journal hints — last 5 from journal-hint stream (v2.5.1)
Vanilla JS + CSS grid. No React. Mermaid (v2.7.1 dep) for Flow viz. Lightweight charts via <canvas> (no Chart.js dep to keep size small).
- Aggregator output shape pinned (3 tests)
- WS event emission frequency throttled (1 test)
- Render snapshot tests using Playwright or similar (4 tests)
uv run pytest tests/test_v27_activity_viz.py -v
# Manual: open Telemetry tab during active session, verify all 8 modules populateMark completed or deferred_to_v2.8.
Status: not_started
Branch suggestion: claude/v2.7.5-skill-split
Tox rebuild: API only
Depends on: v2.6.2 (skill loader exists) + v2.6.1 (hybrid retrieval verifying skills work in production for ≥1 release).
v2.6.2 ships skill packs as ADDITIVE (loaded skills augment SYSTEM_PROMPT_BASE). v2.7.5 finishes the migration: DOMAIN content (POPx, shader, audio-reactive guidance) is MOVED out of SYSTEM_PROMPT_BASE into skill packs. Cleaner always-on prompt → better prefix-cache hits → cheaper turns.
ls src/td_mcp/skills/builtin/ # 6 skills shipped in v2.6.2
git tag --list "v2.6.0*" # v2.6 stable for ≥1 releasetd_component/tdpilot_api_runtime.py::SYSTEM_PROMPT_BASE— remove DOMAIN sections (POPx, shader, audio-reactive, debugging tactics, Python extension patterns). Keep CORE (protocol, intent gate, cycle-detect, output format, tool reference index).src/td_mcp/skills/builtin/<name>/SKILL.md— append any content removed from SYSTEM_PROMPT_BASE- Skill auto-load thresholds — re-tune since base is leaner now (more aggressive auto-load OK)
CHANGELOG.md— describe behavior change
- Existing prompts that previously triggered domain knowledge directly will now require skill load. Auto-load (v2.6.2) covers most cases via keyword matching. Edge cases: subtle prompts that hint at POPx without saying "POP" — may need manual skill load.
- Mitigate via fallback: if no skills load and prompt looks technical (≥40 chars), inject a "general TouchDesigner expert" skill (new built-in for v2.7.5) as the always-loaded floor.
src/td_mcp/skills/builtin/general_td/SKILL.md— fallback skill containing the most-load-bearing domain points that were inSYSTEM_PROMPT_BASE
- Existing
test_v26_skills.py+ new tests:SYSTEM_PROMPT_BASEtoken count drops below threshold (e.g., ≤1200 tokens)- All previous
SYSTEM_PROMPT_BASEcontent surface-area preserved in some skill (audit test) - General-TD fallback loads when no other skill matches
uv run pytest tests/test_v26_skills.py tests/test_v27_skill_split.py -v
# Manual: run a curated set of 20 representative prompts, verify auto-load picks right skill ≥18/20 timesMark completed. v2.7 done.
Same 7 manifests as v2.5/v2.6. All to 2.7.0.
v2.7 adds: td_flow_start, td_flow_status, td_flow_transition, td_flow_abort, td_flow_export_mermaid, td_flow_list_templates (×6 from Flow FSM) + td_self_update (×1) + td_mcp_list_servers, td_mcp_invoke, td_mcp_add_server (×3) = +10 MCP tools. Bump EXPECTED_MIN_TOOL_COUNT: int = 124 (assuming v2.6 hit 114).
Both .tox files need rebuild (chat-pipe gets Flow FSM + skill-split + viz; MCP gets self-update + MCP-consumer).
Lead with "Flow Controller FSM — your agent commits to a plan and shows you the trace" — marquee feature. Then self-update (closes the v2.5.7 loop). Then MCP Config (cross-app interop).
With ~124 MCP tools, plugin description text needs to fit Claude Code marketplace constraints. Verify char limit (~280 in marketplace.json description field).
| Risk | Phase | Probability | Impact | Mitigation |
|---|---|---|---|---|
| Flow FSM rigidity frustrates users | 2.7.1 | High | Med | Always-exitable via abort; opt-in; auto-start opt-out per session |
| Self-update + .tox drift breaks projects | 2.7.2 | Low | High | Source-hash check + snapshot + confirm_token |
| Agent-driven auto-apply of update | 2.7.2 | Low | High | confirm_token HMAC + Tool Approval gate composition |
| External MCP server crashes mid-turn | 2.7.3 | Med | Med | Circuit breaker (v2.4) extension; graceful fallback |
| Tier-allowlist spoofing | 2.7.3 | Low | High | Exact-match; no globs |
| Activity Viz CDN unavailable | 2.7.4 | Low | Low | Vendor Mermaid + canvas-based charts (no Chart.js dep) |
| System-prompt split regresses domain coverage | 2.7.5 | Med | Med | General-TD fallback skill + curated regression suite |
| Schedule slip (Flow FSM bigger than estimated) | 2.7.1 | Med | High | Drop v2.7.4 Activity Viz first; ship Flow + self-update + MCP-config as v2.7.0; viz as v2.7.1 patch |
Same convention as v2.5 §12.
- Before v2.7.1 Flow FSM: verify Mermaid CDN strategy (CDN vs vendored). Default: vendor Mermaid 10.6 to
td_component/static/to avoid air-gapped break. - Before v2.7.2 self-update: decide whether confirm_token uses HMAC (server-side secret) or just timestamp+random. Recommended: HMAC for replay protection.
- Before v2.7.5 skill-split: require ≥1 release of v2.6.2 skill packs in production with no regression reports.
End of v2.7 plan. This concludes the v2.5–v2.7 multi-release roadmap.
After v2.7 ships, evaluate next themes:
- v2.8 candidate: Knowledge Graph retriever (deferred from v2.6 audit)
- v3.0 candidate: OOP worker process, operator-registered tools, local-model support (per original ROADMAP.md Phase 6)
- Continuous: Codex follow-up patches per the established PR pattern