Skip to content

Commit 0764b9f

Browse files
dreamrecclaude
dreamrec
and
claude
committed
ci: add tdpilot_API.tox freshness gate (parallel to dpsk4)
Pre-2.1.1 only tdpilot-dpsk4.tox had a freshness gate; the sibling tdpilot_API.tox went stale silently if a contributor edited the API source tree without rebuilding inside TD — end users would install the plugin and get an old binary while CI stayed green. Mirrors the existing dpsk4 design: build script writes a manifest at rebuild time; CI computes the same hash from the current tree and fails if they don't match. Same JSON format, separate file (.tox-api-source-hash.json), separate gate script. Source list covers: - All direct textDAT embeds from _SOURCE_FILES (skipping the <COMPOSE> sentinel — its content is tracked via the callbacks/ files below). - The 16 files of the callbacks/ split package (their composed body is baked as the mcp_webserver_callbacks textDAT). - The build script itself — any layout change forces a rebuild signal even when no embedded source changed (same reasoning as the dpsk4 gate). Locally verified the gate's three states: fresh -> passes silently source edit -> exits 1 with hash mismatch + rebuild instructions source revert -> passes again Initial hash baked from current v2.1.1 source state: 2bc50e8ebac1fbda... Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 6e4a465 commit 0764b9f

5 files changed

Lines changed: 324 additions & 0 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,9 @@ jobs:
4242
- name: .tox freshness check
4343
run: uv run python scripts/check_tox_freshness.py
4444

45+
- name: API .tox freshness check
46+
run: uv run python scripts/check_tox_api_freshness.py
47+
4548
# Static script invocation — no GitHub event data consumed, so no
4649
# injection risk from user-controlled inputs. The script greps tracked
4750
# files for /Users/<name>/ and C:\Users\<name>\ patterns.

‎CHANGELOG.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,31 @@ lock each piece (palette declared, thick rule, gradient bg, white
8585
body, white-on-red stamp, white brackets, no leakage to other
8686
roles).
8787

88+
### Build/CI — API .tox freshness gate
89+
90+
Pre-2.1.1 only `tdpilot-dpsk4.tox` had a CI freshness gate; its
91+
sibling `tdpilot_API.tox` (which embeds `tdpilot_api_*.py`,
92+
`tdpilot_api_chat.html`, the composed `mcp_webserver_callbacks`
93+
text, and the build script bytes) went stale silently if a
94+
contributor edited the API source tree without rebuilding inside
95+
TD. End users would install the plugin and get an old binary
96+
while CI stayed green.
97+
98+
- `td_component/build_tdpilot_api_tox.py` now writes
99+
`td_component/.tox-api-source-hash.json` at the end of every
100+
rebuild, mirroring the dpsk4 hash file. Hash inputs:
101+
- All direct embeds from `_SOURCE_FILES` (the `<COMPOSE>`
102+
sentinel is skipped — its content is tracked via the
103+
callbacks/ files below).
104+
- The 16 files of the `callbacks/` split package (composed into
105+
the `mcp_webserver_callbacks` textDAT body).
106+
- The build script itself, so any change to .tox layout forces
107+
a rebuild signal even when no embedded source changed.
108+
- New `scripts/check_tox_api_freshness.py` — parallel to the
109+
existing `check_tox_freshness.py`. Same algorithm, different
110+
source list. CI runs it as the new "API .tox freshness check"
111+
step right after the dpsk4 gate.
112+
88113
### Tests
89114

90115
- `tests/test_paused_td_warning.py` — four regression tests
@@ -97,6 +122,18 @@ roles).
97122
- `tests/test_chat_html_v211_user_mark.py` — seven HTML/CSS
98123
pin-tests for the new user-message stamp design.
99124

125+
### Verifying the gate locally
126+
127+
```bash
128+
# Should pass if everything's in sync:
129+
uv run python scripts/check_tox_api_freshness.py
130+
131+
# Should fail with a hash mismatch when source has drifted:
132+
echo "# noqa" >> td_component/tdpilot_api_runtime.py
133+
uv run python scripts/check_tox_api_freshness.py # exit 1
134+
git checkout -- td_component/tdpilot_api_runtime.py
135+
```
136+
100137
## 2.1.0 - 2026-05-08
101138

102139
**Chat UI rework + 13 v2.0 audit fixes.** Collapses what was going

‎scripts/check_tox_api_freshness.py‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
#!/usr/bin/env python3
2+
"""Fail if the committed ``td_component/tdpilot_API.tox`` is out of sync with source.
3+
4+
Parallel to ``scripts/check_tox_freshness.py`` (which guards
5+
``tdpilot-dpsk4.tox``). The .tox is a binary TD artifact and can only
6+
be rebuilt inside TouchDesigner. Pre-2.1.1 only the dpsk4 .tox had a
7+
CI gate, so the API .tox went stale silently if a contributor edited
8+
``tdpilot_api_runtime.py``, ``tdpilot_api_chat.html``, or any other
9+
file under the API source tree without rebuilding inside TD. Users
10+
installing the plugin would get an old binary while CI stayed green.
11+
12+
This guard compares the hash of the source files against the hash
13+
recorded in ``td_component/.tox-api-source-hash.json`` at build time.
14+
Mismatch -> "rebuild the API .tox in TD before pushing".
15+
16+
Runs in CI. Also runnable locally.
17+
"""
18+
19+
from __future__ import annotations
20+
21+
import hashlib
22+
import json
23+
import sys
24+
from pathlib import Path
25+
26+
ROOT = Path(__file__).resolve().parents[1]
27+
28+
# Must match _API_TOX_SOURCE_FILES in td_component/build_tdpilot_api_tox.py.
29+
# Add new files in BOTH places — the build script's list drives the hash
30+
# that gets written; this list drives the check that compares to it.
31+
# A drift between the two lists shows up as a stable mismatch even with
32+
# no source edits (different inputs -> different hashes).
33+
SOURCE_FILES = (
34+
# Direct embeds — every file listed in the build script's _SOURCE_FILES
35+
# tuple as a real path (i.e. not the `<COMPOSE>` sentinel).
36+
"td_component/tdpilot_api_agent.py",
37+
"td_component/tdpilot_api_dispatcher.py",
38+
"td_component/tdpilot_api_config.py",
39+
"td_component/tdpilot_api_lookup.py",
40+
"td_component/tdpilot_api_schema_defs.py",
41+
"td_component/tdpilot_api_schema_map.py",
42+
"td_component/tdpilot_api_schema.py",
43+
"td_component/tdpilot_api_runtime.py",
44+
"td_component/tdpilot_api_extension.py",
45+
"td_component/tdpilot_api_bm25.py",
46+
"td_component/tdpilot_api_memory.py",
47+
"td_component/tdpilot_api_knowledge.py",
48+
"td_component/tdpilot_api_recipes.py",
49+
"td_component/tdpilot_api_skills.py",
50+
"td_component/tdpilot_api_patches.py",
51+
"td_component/tdpilot_api_user_tools.py",
52+
"td_component/tdpilot_api_subagents.py",
53+
"td_component/tdpilot_api_macros.py",
54+
"td_component/tdpilot_api_official_docs.py",
55+
"td_component/tdpilot_api_td2025.py",
56+
"td_component/tdpilot_api_introspect.py",
57+
"td_component/tdpilot_api_batch.py",
58+
"td_component/tdpilot_api_recovery.py",
59+
"td_component/tdpilot_api_tracing.py",
60+
"td_component/tdpilot_api_compaction.py",
61+
"td_component/tdpilot_api_chat.html",
62+
"td_component/tdpilot_api_web_callbacks.py",
63+
"td_component/tdpilot_api_executor.py",
64+
"td_component/tdpilot_api_parexec.py",
65+
# Composed mcp_webserver_callbacks textDAT — its body comes from the
66+
# callbacks/ split package (overlaps with check_tox_freshness.py by
67+
# design; both .tox files embed this composed content).
68+
"td_component/callbacks/_composer.py",
69+
"td_component/callbacks/__init__.py",
70+
"td_component/callbacks/_header.py",
71+
"td_component/callbacks/router.py",
72+
"td_component/callbacks/auth.py",
73+
"td_component/callbacks/serializers.py",
74+
"td_component/callbacks/handlers/__init__.py",
75+
"td_component/callbacks/handlers/nodes.py",
76+
"td_component/callbacks/handlers/exec_and_custom_params.py",
77+
"td_component/callbacks/handlers/exec_python.py",
78+
"td_component/callbacks/handlers/inspect.py",
79+
"td_component/callbacks/handlers/search.py",
80+
"td_component/callbacks/handlers/lifecycle.py",
81+
"td_component/callbacks/handlers/pulse.py",
82+
"td_component/callbacks/handlers/monitor.py",
83+
"td_component/callbacks/handlers/analyze_frame.py",
84+
# Build script bytes — same reasoning as the dpsk4 gate (any change to
85+
# how the .tox is laid out forces a rebuild signal even if no embedded
86+
# source changed).
87+
"td_component/build_tdpilot_api_tox.py",
88+
)
89+
HASH_FILE = ROOT / "td_component" / ".tox-api-source-hash.json"
90+
TOX_FILE = ROOT / "td_component" / "tdpilot_API.tox"
91+
92+
93+
def compute_current_hash() -> str:
94+
h = hashlib.sha256()
95+
for rel in SOURCE_FILES:
96+
path = ROOT / rel
97+
if not path.exists():
98+
continue
99+
h.update(rel.encode("utf-8"))
100+
h.update(b"\x00")
101+
h.update(path.read_bytes())
102+
h.update(b"\x00")
103+
return h.hexdigest()
104+
105+
106+
def main() -> int:
107+
if not TOX_FILE.exists():
108+
print("ERROR: " + str(TOX_FILE.relative_to(ROOT)) + " is missing.")
109+
print(" Rebuild it inside TouchDesigner via")
110+
print(" td_component/build_tdpilot_api_tox.py in the Textport.")
111+
return 1
112+
113+
if not HASH_FILE.exists():
114+
print("ERROR: " + str(HASH_FILE.relative_to(ROOT)) + " is missing.")
115+
print(" Rebuild the API .tox in TouchDesigner — the rebuild writes this file.")
116+
return 1
117+
118+
stored = json.loads(HASH_FILE.read_text())
119+
stored_hash = stored.get("tox_source_hash")
120+
current_hash = compute_current_hash()
121+
122+
if stored_hash != current_hash:
123+
print("ERROR: tdpilot_API.tox is stale relative to td_component source.")
124+
print(" stored hash: " + str(stored_hash))
125+
print(" current hash: " + current_hash)
126+
print(" built at: " + str(stored.get("built_at")))
127+
print("")
128+
print("Rebuild the API .tox in TouchDesigner before pushing.")
129+
print(" See td_component/build_tdpilot_api_tox.py and the Textport recipe in")
130+
print(" feedback_td_tox_rebuild_recipe.md (single-line statements only).")
131+
print(" Then: git add td_component/tdpilot_API.tox td_component/.tox-api-source-hash.json")
132+
return 1
133+
134+
print(
135+
"tdpilot_API.tox is fresh (hash "
136+
+ current_hash[:16]
137+
+ "..., built "
138+
+ str(stored.get("built_at"))
139+
+ ")"
140+
)
141+
return 0
142+
143+
144+
if __name__ == "__main__":
145+
sys.exit(main())

‎td_component/.tox-api-source-hash.json‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
{
2+
"tox_source_hash": "2bc50e8ebac1fbda1168d3bfbb7dfc8e6e389cc2fc24a05784505491d692ee86",
3+
"built_at": "2026-05-08T20:29:30.074468+00:00",
4+
"source_files": [
5+
"td_component/tdpilot_api_agent.py",
6+
"td_component/tdpilot_api_dispatcher.py",
7+
"td_component/tdpilot_api_config.py",
8+
"td_component/tdpilot_api_lookup.py",
9+
"td_component/tdpilot_api_schema_defs.py",
10+
"td_component/tdpilot_api_schema_map.py",
11+
"td_component/tdpilot_api_schema.py",
12+
"td_component/tdpilot_api_runtime.py",
13+
"td_component/tdpilot_api_extension.py",
14+
"td_component/tdpilot_api_bm25.py",
15+
"td_component/tdpilot_api_memory.py",
16+
"td_component/tdpilot_api_knowledge.py",
17+
"td_component/tdpilot_api_recipes.py",
18+
"td_component/tdpilot_api_skills.py",
19+
"td_component/tdpilot_api_patches.py",
20+
"td_component/tdpilot_api_user_tools.py",
21+
"td_component/tdpilot_api_subagents.py",
22+
"td_component/tdpilot_api_macros.py",
23+
"td_component/tdpilot_api_official_docs.py",
24+
"td_component/tdpilot_api_td2025.py",
25+
"td_component/tdpilot_api_introspect.py",
26+
"td_component/tdpilot_api_batch.py",
27+
"td_component/tdpilot_api_recovery.py",
28+
"td_component/tdpilot_api_tracing.py",
29+
"td_component/tdpilot_api_compaction.py",
30+
"td_component/tdpilot_api_chat.html",
31+
"td_component/tdpilot_api_web_callbacks.py",
32+
"td_component/tdpilot_api_executor.py",
33+
"td_component/tdpilot_api_parexec.py",
34+
"td_component/callbacks/_composer.py",
35+
"td_component/callbacks/__init__.py",
36+
"td_component/callbacks/_header.py",
37+
"td_component/callbacks/router.py",
38+
"td_component/callbacks/auth.py",
39+
"td_component/callbacks/serializers.py",
40+
"td_component/callbacks/handlers/__init__.py",
41+
"td_component/callbacks/handlers/nodes.py",
42+
"td_component/callbacks/handlers/exec_and_custom_params.py",
43+
"td_component/callbacks/handlers/exec_python.py",
44+
"td_component/callbacks/handlers/inspect.py",
45+
"td_component/callbacks/handlers/search.py",
46+
"td_component/callbacks/handlers/lifecycle.py",
47+
"td_component/callbacks/handlers/pulse.py",
48+
"td_component/callbacks/handlers/monitor.py",
49+
"td_component/callbacks/handlers/analyze_frame.py",
50+
"td_component/build_tdpilot_api_tox.py"
51+
]
52+
}

‎td_component/build_tdpilot_api_tox.py‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@
3535

3636
from __future__ import annotations
3737

38+
import hashlib
39+
import json
3840
import os
3941
import sys
4042
from datetime import datetime, timezone
@@ -219,6 +221,88 @@ def _load_legacy_module():
219221
)
220222

221223

224+
# ---------------------------------------------------------------------------
225+
# v2.1.1 — API .tox freshness tracking (parallel to the dpsk4 .tox gate
226+
# in scripts/check_tox_freshness.py).
227+
#
228+
# Why: pre-2.1.1 only tdpilot-dpsk4.tox had a CI gate; tdpilot_API.tox
229+
# went stale silently if a contributor edited runtime.py / chat.html /
230+
# anywhere in the API source tree without rebuilding inside TD. Users
231+
# installing the plugin would get an old binary while CI stayed green.
232+
#
233+
# How: at build time we compute a sha256 over the byte content of every
234+
# source file that contributes to the API .tox, and write it alongside
235+
# the existing dpsk4 hash file under td_component/.tox-api-source-hash.json.
236+
# scripts/check_tox_api_freshness.py recomputes the hash from the current
237+
# tree and fails CI if it doesn't match what's stored.
238+
# ---------------------------------------------------------------------------
239+
240+
241+
# Direct embeds — pulled from _SOURCE_FILES (skip the `<COMPOSE>` sentinel
242+
# whose content comes from the callbacks/ split package below).
243+
_API_TOX_SOURCE_FILES = tuple(rel for _, _, rel in _SOURCE_FILES if rel != "<COMPOSE>") + (
244+
# Composed mcp_webserver_callbacks textDAT pulls its body from the
245+
# callbacks/ split package via _legacy._read_callbacks_source(). Any
246+
# byte change in any of these files changes the composed text that
247+
# gets baked into the API .tox, so the hash must cover them too.
248+
# NOTE: this list overlaps with check_tox_freshness.py's SOURCE_FILES
249+
# by design — both .tox files embed the composed callbacks body.
250+
"td_component/callbacks/_composer.py",
251+
"td_component/callbacks/__init__.py",
252+
"td_component/callbacks/_header.py",
253+
"td_component/callbacks/router.py",
254+
"td_component/callbacks/auth.py",
255+
"td_component/callbacks/serializers.py",
256+
"td_component/callbacks/handlers/__init__.py",
257+
"td_component/callbacks/handlers/nodes.py",
258+
"td_component/callbacks/handlers/exec_and_custom_params.py",
259+
"td_component/callbacks/handlers/exec_python.py",
260+
"td_component/callbacks/handlers/inspect.py",
261+
"td_component/callbacks/handlers/search.py",
262+
"td_component/callbacks/handlers/lifecycle.py",
263+
"td_component/callbacks/handlers/pulse.py",
264+
"td_component/callbacks/handlers/monitor.py",
265+
"td_component/callbacks/handlers/analyze_frame.py",
266+
# Build script bytes — same reasoning as the dpsk4 gate (any change
267+
# to how the .tox is laid out forces a rebuild signal even if no
268+
# embedded source changed).
269+
"td_component/build_tdpilot_api_tox.py",
270+
)
271+
272+
273+
def _compute_api_tox_source_hash(repo_root):
274+
"""Return sha256 over the bytes of every file that feeds tdpilot_API.tox.
275+
276+
Single source of truth for API .tox freshness. The matching list
277+
in scripts/check_tox_api_freshness.py must stay aligned — a comment
278+
in each file points at the other.
279+
"""
280+
h = hashlib.sha256()
281+
for rel in _API_TOX_SOURCE_FILES:
282+
path = os.path.join(repo_root, rel)
283+
if not os.path.isfile(path):
284+
continue
285+
h.update(rel.encode("utf-8"))
286+
h.update(b"\x00")
287+
with open(path, "rb") as f:
288+
h.update(f.read())
289+
h.update(b"\x00")
290+
return h.hexdigest()
291+
292+
293+
def _write_api_tox_source_hash(repo_root):
294+
"""Record the API .tox source hash so CI can detect drift after edits."""
295+
manifest = {
296+
"tox_source_hash": _compute_api_tox_source_hash(repo_root),
297+
"built_at": datetime.now(timezone.utc).isoformat(),
298+
"source_files": list(_API_TOX_SOURCE_FILES),
299+
}
300+
out_path = os.path.join(repo_root, "td_component", ".tox-api-source-hash.json")
301+
with open(out_path, "w", encoding="utf-8") as f:
302+
json.dump(manifest, f, indent=2)
303+
print(f"[tdpilot_API] Wrote {out_path}")
304+
305+
222306
# ---------------------------------------------------------------------------
223307
# Custom-param helpers
224308
# ---------------------------------------------------------------------------
@@ -753,6 +837,9 @@ def build_and_export():
753837
version = _legacy._get_version(repo_root)
754838
print(f"[tdpilot_API] built v{version}")
755839
print(f"[tdpilot_API] exported TOX: {export_path}")
840+
# v2.1.1 — refresh the API .tox source-hash manifest so CI's
841+
# check_tox_api_freshness gate stays green.
842+
_write_api_tox_source_hash(repo_root)
756843
if install_parent is None:
757844
print(f"[tdpilot_API] drag {export_path} into a TD project to install.")
758845
return export_path

0 commit comments

Comments
 (0)