Skip to content

feat(api-tox): v2.2.0 Phase 1.2 — cycle detection in tool chains - #37

Merged
dreamrec merged 2 commits into
mainfrom
claude/v2.2.0-feature-1.2-cycle-detection
May 11, 2026
Merged

feat(api-tox): v2.2.0 Phase 1.2 — cycle detection in tool chains#37
dreamrec merged 2 commits into
mainfrom
claude/v2.2.0-feature-1.2-cycle-detection

Conversation

@dreamrec

Copy link
Copy Markdown
Owner

Summary

Second feature of the v2.2.0 reliability phase (1.1 was auto-rollback, shipped in #34 + Codex-followup #36; consolidation pre-work in #35). 1.2 pairs with 1.1 to cover the second high-impact reliability gap:

Failure mode Caught by
"agent broke things" 1.1 auto-rollback (undo the batch)
"agent is stuck looping" 1.2 cycle detection (break the turn)

Per-turn ledger tracks (tool_name, args_hash) -> count. Before each dispatch, ledger.record(name, args) increments; when the count reaches the threshold (default 3), CycleDetected raises BEFORE invoking the tool. run_turn's BaseException catch fires on_errorEV_ERROR + EV_STATE: idle → red banner in the chat UI carrying the stuck tool's name + an args preview.

No version bump — Phase 1 ships as v2.2.0 only when the whole phase is in.

Design highlights

Decision Rationale
Threshold = 3 (configurable) 2 dispatches allowed before the 3rd attempt blocks. One re-attempt covers transient blips (DeepSeek 5xx retry, momentary cook conflict) without papering over real loops.
Args hash via sorted-keys JSON {"a":1,"b":2} and {"b":2,"a":1} collapse to the same key; nested dicts hash recursively; lists are order-sensitive (connect source/dest order matters).
Check BEFORE on_tool_call A blocked call shouldn't leak EV_TOOL_CALL that the chat UI then has to retract a tick later. Cleaner UI state machine.
Per-turn lifetime Built once at _loop start, discarded at turn end. Identical calls between turns aren't a cycle — that's the user re-asking.
Late-import CycleDetected in _loop cycle_detector imports AgentError from agent. Top-level import in agent → circular. Late import inside _loop resolves it; Python's module cache makes cost = 1 dict lookup per turn after first call.
Subclass of AgentError Same family as TurnBudgetExceeded. The existing run_turn catch-all (except BaseException) routes it through on_error automatically; no new event wiring needed.

Files

New (2):

  • td_component/tdpilot_api_cycle_detector.py — ~210 lines, pure-Python. Exports CycleDetected exception, CycleLedger class, args_hash, env-var gate, factory builder.
  • tests/test_tdpilot_api_cycle_detector.py — ~330 lines, 49 tests across 8 classes.

Modified (4):

  • td_component/tdpilot_api_agent.py — new cycle_ledger_factory ctor kwarg; _loop builds one ledger per turn; inner per-batch loop checks ledger.record() BEFORE on_tool_call/dispatch; raises at threshold.
  • td_component/tdpilot_api_runtime.py — new _build_cycle_ledger_factory method; honours TDPILOT_DISABLE_CYCLE_DETECTION=1; passes factory to Agent().
  • td_component/build_tdpilot_api_tox.py — adds tdpilot_api_cycle_detector to _SOURCE_FILES. Post-refactor: consolidate paired .tox source-file lists #35 the freshness gate auto-imports this list, so no paired-list maintenance.
  • CHANGELOG.md — new "Feature 1.2" entry under "Unreleased".

Binary update (2): td_component/tdpilot_API.tox (221,182 → 222,406 → now 222,406 + ~1.5KB cycle-detector textDAT) + .tox-api-source-hash.json.

Tests (49 new)

Class Tests What it pins
TestArgsHash 8 Order independence; None/{} normalisation; nested-dict ordering; list-order sensitivity; compact-separators invariant; defensive paths for non-JSON values.
TestCycleLedger 10 Threshold validation (rejects <2); increment correctness; per-key isolation; peek is idempotent; reset clears; __len__ semantics.
TestCycleDetectedException 3 Carries tool_name/count/args_summary; message formatting; isinstance(AgentError) so run_turn catches it.
TestEnvVarGate 3 Truthy / falsy / unset env-var classification.
TestBuildCycleLedgerFactory 3 Env-disable returns None; env-enabled returns callable producing fresh per-turn instances; custom threshold propagates.
TestUsagePattern 4 Canonical check-then-dispatch flow; 2 identical calls don't trigger; alternating calls eventually trigger (proves ledger spans the whole turn).
TestFormatArgsSummary 3 Empty/short/long args rendering.
TestAgentLoopCycleIntegration 3 End-to-end with mocked urlopen: 3 identical calls → raises on 3rd, dispatches only 2, fires on_error; disabled factory → 4 dispatches hit turn budget; distinct args → completes cleanly without triggering.

Test plan

  • uv run pytest tests/ --ignore=tests/agent_evals -q1821 passed (1772 prior + 49 new).
  • uv run --extra dev ruff format --check src tests scripts td_component — clean.
  • uv run --extra dev ruff check src tests scripts td_component — clean.
  • uv run python scripts/check_versions.py — in sync at v2.1.5.
  • uv run python scripts/check_tox_freshness.py — dpsk4 .tox fresh (not touched).
  • uv run python scripts/check_tox_api_freshness.py — API .tox fresh post-rebuild (hash 0f6a0f482630993d...).
  • bash scripts/check_no_personal_paths.sh — clean.
  • CI all-green on push.

Manual verification (post-merge, in TD)

  1. Open tdpilot_API.tox in /project1.
  2. Ask the agent something where it will likely loop (e.g. an ambiguous "what's wrong with my project" while the project has a misleading error state).
  3. Expected: after 2 identical lookups, the 3rd attempt is blocked; chat shows a red banner reading "Cycle detected: td_get_errors ×3 with identical args (…)" and the turn ends with EV_STATE: idle.
  4. Disable check: set TDPILOT_DISABLE_CYCLE_DETECTION=1, reload the COMP, repeat — agent should now loop until the turn budget exhausts.

Why mock-only and not live-TD verification

User's TD has TD_MCP_REQUIRE_AUTH=1 so my MCP client hits 401, blocking live verification through the dpsk4 MCP server. The 3 integration tests in TestAgentLoopCycleIntegration cover the agent loop's late-import + raise + on_error path with a real Agent instance + mocked urlopen — high-fidelity coverage. If anything slips, the standing Codex-followup pattern catches it post-merge.

🤖 Generated with Claude Code

dreamrec and others added 2 commits May 11, 2026 17:50
Second feature of the v2.2.0 reliability phase (Phase 1.1 was
auto-rollback; see PR #34 + Codex P1/P2 follow-up in PR #36). Pairs
with 1.1 to cover the two high-impact reliability gaps:

  - 1.1 catches "agent broke things" → undo the batch
  - 1.2 catches "agent is stuck looping" → break the turn

How it works
============

A ``CycleLedger`` is constructed once per turn in ``Agent._loop``
and tracks ``(tool_name, args_hash) -> count``. Each call to
``ledger.record(name, args)`` increments and returns the new count.
The agent loop checks against the threshold (default 3) BEFORE
dispatching:

  - count == 1: first time, dispatch.
  - count == 2: second time, dispatch.
  - count >= 3: ``CycleDetected`` raised; ``run_turn``'s catch fires
    ``on_error`` → ``EV_ERROR`` + ``EV_STATE: idle`` in the event
    stream → red banner in the chat UI with tool name + args preview.

The threshold of 3 is conservative on purpose — 2 dispatches give
the agent a chance to recover from a one-off failure before declaring
the turn stuck.

Args identity uses sorted-keys JSON so ``{"a":1,"b":2}`` and
``{"b":2,"a":1}`` collapse to the same key; nested dicts hash
recursively; lists are order-sensitive (connect source/dest
ordering matters). Disable via env var
``TDPILOT_DISABLE_CYCLE_DETECTION=1``.

Implementation
==============

- NEW td_component/tdpilot_api_cycle_detector.py — pure-Python:
  args_hash, CycleLedger, CycleDetected, env-var gate, factory
  builder. No TouchDesigner imports.

- tdpilot_api_agent.py:
  - New cycle_ledger_factory ctor kwarg.
  - _loop builds one ledger per turn after model resolution.
  - Inner per-batch for-loop checks ledger.record() BEFORE
    on_tool_call/dispatch; raises CycleDetected at the threshold.
  - CycleDetected is **late-imported** from cycle_detector inside
    _loop to avoid a circular import (cycle_detector imports
    AgentError from agent at top-level). Cost: one dict lookup per
    turn after the import is cached.

- tdpilot_api_runtime.py:
  - New _build_cycle_ledger_factory method honours
    TDPILOT_DISABLE_CYCLE_DETECTION=1.
  - AgentRuntime._build_agent passes the factory to Agent().

- build_tdpilot_api_tox.py:
  - Adds tdpilot_api_cycle_detector to _SOURCE_FILES so it's baked
    into the API .tox. (Post-#35 the freshness gate auto-imports
    this same list, so no parallel-list maintenance.)

Tests (49 new)
==============

- TestArgsHash (8) — order independence, normalization, nested
  ordering, list-order sensitivity, defensive paths.
- TestCycleLedger (10) — threshold validation (>=2), increment,
  per-key isolation, peek/reset, len semantics.
- TestCycleDetectedException (3) — metadata, message formatting,
  AgentError membership.
- TestEnvVarGate (3) — truthy/falsy classification.
- TestBuildCycleLedgerFactory (3) — env-disable, fresh instances,
  custom threshold.
- TestUsagePattern (4) — the canonical check-then-dispatch flow.
- TestFormatArgsSummary (3) — render rules.
- TestAgentLoopCycleIntegration (3) — end-to-end with mocked
  urlopen, proving the late-import + raise + on_error path
  actually works in run_turn. Covers: enabled+stuck (raises on
  3rd, dispatches 2), disabled-factory (no check, hits
  turn_budget), distinct-args (never triggers, completes cleanly).

Local sweep
===========

- pytest: 1821 passed (1772 prior + 49 new).
- ruff format + check: clean.
- check_versions: in sync at v2.1.5 (no bump on this PR).
- check_tox_freshness (dpsk4): fresh — that .tox not touched.
- check_tox_api_freshness (chat-pipe): EXPECTED FAIL until the
  user rebuilds the .tox via the canonical Textport recipe.
- check_no_personal_paths: clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dreamrec
dreamrec merged commit d41ac70 into main May 11, 2026
6 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88532a9235

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +176 to +179
key = (str(tool_name), args_hash(args))
new_count = self._counts.get(key, 0) + 1
self._counts[key] = new_count
return new_count

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Count only consecutive repeats before declaring a cycle

This ledger increments per (tool_name, args_hash) across the entire turn and never resets when a different call happens, so non-looping patterns like td_get_errors between unrelated mutation steps still trip at the 3rd check. In normal validate/fix/validate flows, the turn will be aborted even though the agent is progressing, which creates false-positive CycleDetected failures and prematurely ends useful runs.

Useful? React with 👍 / 👎.

Comment on lines +621 to +626
if count >= cycle_ledger.threshold:
raise CycleDetected(
tool_name=tool_name,
count=count,
args_summary=_format_cycle_args(tool_args),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve protocol invariants when blocking a tool call

Raising CycleDetected here exits before a tool_result is appended for the current assistant tool_use, leaving conversation history with a dangling tool_use block. After this error, the next turn reuses that invalid transcript and can be rejected by the Anthropic/DeepSeek tool-use protocol (missing matching tool_result), so one cycle-detection event can poison subsequent requests until history is repaired.

Useful? React with 👍 / 👎.

dreamrec added a commit that referenced this pull request May 11, 2026
First milestone of the v2.2.0→v3.0 roadmap (see docs/ROADMAP.md).
Phase 1 ships in full; the chat-pipe agent is now safe to leave
unsupervised on complex builds, two failure modes have automatic
recovery, and drag-and-go finally works for new users out of the box.

Bundled in this release
=======================

  - 1.1 Auto-rollback on error regression (PR #34 + #36)
  - 1.2 Cycle detection in tool chains (PR #37)
  - 1.2.1 Drag-and-go UX polish — Authmode COMP param,
    auto-save+auto-reload on Apikey change, 401 reconnect
    banner (PR #38)
  - 1.2.2 Build scripts auto-mirror .tox into main repo on rebuild,
    eliminating stale-symlink footgun (PR #39)
  - PR #35 paired source-file-list refactor (build-script + freshness-
    check now share a single source of truth)

136 new tests since v2.1.5. Total suite at release: 1848 passing.

Headline behaviour change for end users
=======================================

Default chat-pipe webserver auth posture changes from token-required
to origin-allowlist-only (Authmode=open default). The origin
allowlist still rejects cross-origin browser CSRF, so this is safe
on TouchDesigner's typical single-user dev/perform usage profile.
Users sharing .toe files across machines should set Authmode=token
on the COMP for that deployment.

The MCP-server tdpilot-dpsk4.tox (port 9985) auth model is UNCHANGED
— TD_MCP_SHARED_SECRET still required.

Bumps + rebuilt artifacts
=========================

All 10 versioned files (7 manifests + 3 doc titles) bumped from
2.1.5 to 2.2.0 (scripts/check_versions.py enforces sync). API_VERSION
in td_component/callbacks/_header.py also bumped; this invalidated
both .tox source-hashes, both rebuilt inside TouchDesigner via the
canonical recipe. Auto-mirror (Phase 1.2.2, just landed) propagated
the rebuilt .tox files to the main repo's td_component/ automatically.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dreamrec
dreamrec deleted the claude/v2.2.0-feature-1.2-cycle-detection branch May 11, 2026 22:48
dreamrec added a commit that referenced this pull request May 19, 2026
* feat(api-tox): v2.2.0 Phase 1.2 — cycle detection in tool chains

Second feature of the v2.2.0 reliability phase (Phase 1.1 was
auto-rollback; see PR #34 + Codex P1/P2 follow-up in PR #36). Pairs
with 1.1 to cover the two high-impact reliability gaps:

  - 1.1 catches "agent broke things" → undo the batch
  - 1.2 catches "agent is stuck looping" → break the turn

How it works
============

A ``CycleLedger`` is constructed once per turn in ``Agent._loop``
and tracks ``(tool_name, args_hash) -> count``. Each call to
``ledger.record(name, args)`` increments and returns the new count.
The agent loop checks against the threshold (default 3) BEFORE
dispatching:

  - count == 1: first time, dispatch.
  - count == 2: second time, dispatch.
  - count >= 3: ``CycleDetected`` raised; ``run_turn``'s catch fires
    ``on_error`` → ``EV_ERROR`` + ``EV_STATE: idle`` in the event
    stream → red banner in the chat UI with tool name + args preview.

The threshold of 3 is conservative on purpose — 2 dispatches give
the agent a chance to recover from a one-off failure before declaring
the turn stuck.

Args identity uses sorted-keys JSON so ``{"a":1,"b":2}`` and
``{"b":2,"a":1}`` collapse to the same key; nested dicts hash
recursively; lists are order-sensitive (connect source/dest
ordering matters). Disable via env var
``TDPILOT_DISABLE_CYCLE_DETECTION=1``.

Implementation
==============

- NEW td_component/tdpilot_api_cycle_detector.py — pure-Python:
  args_hash, CycleLedger, CycleDetected, env-var gate, factory
  builder. No TouchDesigner imports.

- tdpilot_api_agent.py:
  - New cycle_ledger_factory ctor kwarg.
  - _loop builds one ledger per turn after model resolution.
  - Inner per-batch for-loop checks ledger.record() BEFORE
    on_tool_call/dispatch; raises CycleDetected at the threshold.
  - CycleDetected is **late-imported** from cycle_detector inside
    _loop to avoid a circular import (cycle_detector imports
    AgentError from agent at top-level). Cost: one dict lookup per
    turn after the import is cached.

- tdpilot_api_runtime.py:
  - New _build_cycle_ledger_factory method honours
    TDPILOT_DISABLE_CYCLE_DETECTION=1.
  - AgentRuntime._build_agent passes the factory to Agent().

- build_tdpilot_api_tox.py:
  - Adds tdpilot_api_cycle_detector to _SOURCE_FILES so it's baked
    into the API .tox. (Post-#35 the freshness gate auto-imports
    this same list, so no parallel-list maintenance.)

Tests (49 new)
==============

- TestArgsHash (8) — order independence, normalization, nested
  ordering, list-order sensitivity, defensive paths.
- TestCycleLedger (10) — threshold validation (>=2), increment,
  per-key isolation, peek/reset, len semantics.
- TestCycleDetectedException (3) — metadata, message formatting,
  AgentError membership.
- TestEnvVarGate (3) — truthy/falsy classification.
- TestBuildCycleLedgerFactory (3) — env-disable, fresh instances,
  custom threshold.
- TestUsagePattern (4) — the canonical check-then-dispatch flow.
- TestFormatArgsSummary (3) — render rules.
- TestAgentLoopCycleIntegration (3) — end-to-end with mocked
  urlopen, proving the late-import + raise + on_error path
  actually works in run_turn. Covers: enabled+stuck (raises on
  3rd, dispatches 2), disabled-factory (no check, hits
  turn_budget), distinct-args (never triggers, completes cleanly).

Local sweep
===========

- pytest: 1821 passed (1772 prior + 49 new).
- ruff format + check: clean.
- check_versions: in sync at v2.1.5 (no bump on this PR).
- check_tox_freshness (dpsk4): fresh — that .tox not touched.
- check_tox_api_freshness (chat-pipe): EXPECTED FAIL until the
  user rebuilds the .tox via the canonical Textport recipe.
- check_no_personal_paths: clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebuild tdpilot_API.tox for Phase 1.2 cycle detection

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dreamrec added a commit that referenced this pull request May 19, 2026
First milestone of the v2.2.0→v3.0 roadmap (see docs/ROADMAP.md).
Phase 1 ships in full; the chat-pipe agent is now safe to leave
unsupervised on complex builds, two failure modes have automatic
recovery, and drag-and-go finally works for new users out of the box.

Bundled in this release
=======================

  - 1.1 Auto-rollback on error regression (PR #34 + #36)
  - 1.2 Cycle detection in tool chains (PR #37)
  - 1.2.1 Drag-and-go UX polish — Authmode COMP param,
    auto-save+auto-reload on Apikey change, 401 reconnect
    banner (PR #38)
  - 1.2.2 Build scripts auto-mirror .tox into main repo on rebuild,
    eliminating stale-symlink footgun (PR #39)
  - PR #35 paired source-file-list refactor (build-script + freshness-
    check now share a single source of truth)

136 new tests since v2.1.5. Total suite at release: 1848 passing.

Headline behaviour change for end users
=======================================

Default chat-pipe webserver auth posture changes from token-required
to origin-allowlist-only (Authmode=open default). The origin
allowlist still rejects cross-origin browser CSRF, so this is safe
on TouchDesigner's typical single-user dev/perform usage profile.
Users sharing .toe files across machines should set Authmode=token
on the COMP for that deployment.

The MCP-server tdpilot-dpsk4.tox (port 9985) auth model is UNCHANGED
— TD_MCP_SHARED_SECRET still required.

Bumps + rebuilt artifacts
=========================

All 10 versioned files (7 manifests + 3 doc titles) bumped from
2.1.5 to 2.2.0 (scripts/check_versions.py enforces sync). API_VERSION
in td_component/callbacks/_header.py also bumped; this invalidated
both .tox source-hashes, both rebuilt inside TouchDesigner via the
canonical recipe. Auto-mirror (Phase 1.2.2, just landed) propagated
the rebuilt .tox files to the main repo's td_component/ automatically.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant