Interactive escalations for the agent-facing run/debug tools by georgi · Pull Request #4639 · nodetool-ai/nodetool

georgi · 2026-08-02T08:06:08Z

What

The supervisor put an LLM on a run's failure path; this puts the calling agent there. The workflow execution and debugging tools now bubble node failures up as escalations and let the agent decide, using the same verdict vocabulary and kernel guarantees:

run_workflow / debug_workflow accept interactive: true. A failing node invocation parks the run and the tool returns status: "escalated" with the supervisor's Escalation record (redacted inputs, error detail, allowedActions).
The agent answers via the new resolve_workflow_escalation tool — retry, substitute, skip, end_stream, or fail — and gets back either the next escalation or the run's final report (the same debug summary + verdict as before).

How

packages/websocket/src/debug-sessions.ts (new): InteractiveEscalationHandle, a SupervisorHandle whose decide() waits for a verdict delivered over HTTP, plus a user-owned session registry (sessions expire 10 min after the run settles).
http-api.ts: interactive (plus max_decisions / max_retries_per_node / decision_timeout_ms) on POST /api/workflows/:id/run|debug; new GET/POST /api/debug/sessions/:id[/verdict|/cancel] routes. The run/finalize/report tail of handleWorkflowRun is factored into helpers shared by the synchronous and interactive paths.
The handle is wrapped in the existing BoundedHandle, so decision/retry caps, sticky (applyTo: "signature") verdicts, and the fail-closed decision timeout all apply unchanged — the timeout defaults to 10 min for an agent's tool round trip instead of 60 s. The kernel still independently enforces allowedActions; the endpoint additionally rejects a disallowed or malformed verdict with a 400 without deciding the escalation, so the agent can correct itself instead of killing the node.
Agent tools export/bridge: ResolveWorkflowEscalationTool added to getAllMcpTools and the MCP server bridge; escalated payloads carry a next_tool hint so the loop is self-describing.

Tests

packages/websocket/tests/debug-interactive.test.ts: full loop through the real kernel (escalate → skip → completed report with the intervention recorded; fail with reason; disallowed/malformed verdicts rejected non-terminally; sessions invisible to other users).
packages/agents/tests/mcp-tools.test.ts: interactive flag forwarding, escalated-response short-circuit in debug_workflow, verdict body shapes for the new tool.
Full suites green: agents (1882) and websocket (2011); root lint clean; typecheck clean except the pre-existing mobile failure (Expo deps not installed in this sandbox, identical on a clean tree).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KNZApg6rgnpV9aqs5BTyX7

Generated by Claude Code

The supervisor put an LLM on a run's failure path; this puts the calling agent there. `run_workflow`/`debug_workflow` accept `interactive: true`: a failing node invocation parks the run and the tool returns the escalation record, the agent answers with a verdict via the new `resolve_workflow_escalation` tool, and the final report arrives once the run settles. - `packages/websocket/src/debug-sessions.ts`: `InteractiveEscalationHandle` (a `SupervisorHandle` whose `decide()` waits for an HTTP verdict) plus a user-owned session registry with a post-run TTL. - `http-api.ts`: `interactive` on `POST /api/workflows/:id/run|debug` (responds with the final report or `status: "escalated"`), new `GET/POST /api/debug/sessions/:id[/verdict|/cancel]` routes. The handle is wrapped in the same `BoundedHandle` (caps, sticky verdicts, fail-closed timeout — sized at 10 min for a tool round trip), and the kernel still enforces `allowedActions`; a disallowed or malformed verdict 400s without deciding the escalation, so the agent can correct itself. - Agent tools: `interactive` flag on run/debug, new `resolve_workflow_escalation`, bridged onto the MCP server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KNZApg6rgnpV9aqs5BTyX7

Copilot

Pull request overview

Adds an “interactive escalation” mode to the agent-facing workflow execution tools so node failures can pause a run and be resolved by the calling agent via a new follow-up tool/HTTP endpoint, instead of failing the whole run immediately.

Changes:

Introduces in-memory interactive debug session tracking (DebugSessionRegistry) and an InteractiveEscalationHandle to park/resume runs via verdicts.
Extends the HTTP run/debug endpoints with interactive options and adds /api/debug/sessions/:id[/verdict|/cancel].
Exposes the new capability through MCP tools (resolve_workflow_escalation) and adds end-to-end tests for the interactive loop.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/websocket/src/debug-sessions.ts	New interactive session + escalation handle implementation used by HTTP run/debug.
packages/websocket/src/http-api.ts	Adds `interactive` mode to run/debug and new debug-session routes; refactors run finalization/payload building.
packages/websocket/src/mcp-agent-tools.ts	Registers the new agent MCP tool (`ResolveWorkflowEscalationTool`) on the websocket server bridge.
packages/agents/src/tools/mcp-tools.ts	Adds `resolve_workflow_escalation` tool + interactive flag forwarding/annotation for escalated runs.
packages/agents/src/index.ts	Exports `ResolveWorkflowEscalationTool`.
packages/websocket/tests/debug-interactive.test.ts	New websocket-layer test covering escalate → verdict → completion/failure + access control.
packages/agents/tests/mcp-tools.test.ts	Tests interactive forwarding and the new resolve tool request/response shapes.
packages/websocket/tests/mcp-server-coverage.test.ts	Ensures the MCP server coverage test includes the new bridged tool.
docs/workflow-supervisor-design.md	Documents the new “interactive (agent tools)” surface.
CLAUDE.md	Updates CLI/agent documentation to include the interactive escalation flow.

+  async waitForEvent(): Promise<DebugSessionEvent> {
+    return Promise.race([
+      this._done.then(
+        (report): DebugSessionEvent => ({ kind: "done", report })
+      ),


+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error);
+      try {
+        job.markFailed(message);
+        await job.save();
+      } catch (saveError) {
+        log.warn("failed to persist failed job status", {
+          jobId: job.id,
+          error: String(saveError)
+        });
+      }
+      return {
+        job_id: job.id,
+        workflow_id: workflowId,
+        status: "failed",
+        error: message
+      };


+      supervisorHandle = new BoundedHandle(interactiveHandle, {
+        ...(typeof body?.max_decisions === "number"
+          ? { maxDecisions: body.max_decisions }
+          : {}),
+        ...(typeof body?.max_retries_per_node === "number"
+          ? { maxRetriesPerNode: body.max_retries_per_node }
+          : {}),
+        decisionTimeoutMs:
+          typeof body?.decision_timeout_ms === "number"
+            ? body.decision_timeout_ms
+            : INTERACTIVE_DECISION_TIMEOUT_MS
+      });


…aiter leak - A thrown interactive run now reports through buildWorkflowRunPayload, so the debug surface keeps its summary + verdict shape on hard failures. - max_decisions / max_retries_per_node / decision_timeout_ms are API inputs: anything but a sane integer falls back to the default instead of reaching BoundedHandle (a NaN timeout failed every decision instantly). - waitForEvent's escalation subscription is cancellable, so the done branch winning the race no longer strands a waiter per verdict round trip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KNZApg6rgnpV9aqs5BTyX7

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (2)

packages/agents/src/tools/mcp-tools.ts:847

ResolveWorkflowEscalationTool can send an invalid verdict for action: "substitute": protocol verdictSchema requires outputs, but the tool omits it when params.outputs is missing. That will reliably 400 at /verdict and forces the model into a trial-and-error loop.

Fix: fail fast in the tool when action === "substitute" and outputs is absent (or make it conditional-required in the JSON schema).

    const action = String(params["action"]);
    const verdict: Record<string, unknown> = { action };
    if (action === "substitute" && params["outputs"] !== undefined) {
      verdict["outputs"] = params["outputs"];
    }

packages/websocket/src/http-api.ts:1142

The 400 guard for /api/debug/sessions/:id/verdict says the body must include both escalation_id and verdict, but it only validates escalation_id. When verdict is missing, the request falls through to verdictSchema.safeParse(undefined) and returns a different 400, which is inconsistent with the intended error.

Fix: include an explicit body.verdict === undefined check in the guard.

    if (!body || typeof body.escalation_id !== "string") {
      return errorResponse(400, "Body must carry escalation_id and verdict");
    }

Copilot AI review requested due to automatic review settings August 2, 2026 08:06

Copilot started reviewing on behalf of georgi August 2, 2026 08:06 View session

Copilot AI reviewed Aug 2, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 2, 2026 08:11

Copilot started reviewing on behalf of georgi August 2, 2026 08:12 View session

georgi enabled auto-merge (squash) August 2, 2026 08:12

Copilot AI reviewed Aug 2, 2026

View reviewed changes

georgi merged commit 8a66625 into main Aug 2, 2026
24 checks passed

georgi deleted the claude/workflow-execution-debugging-n9uea9 branch August 2, 2026 08:24

georgi mentioned this pull request Aug 3, 2026

marketing: reposition the agents page as agent-first #4665

Merged

Navigation Menu

Uh oh!

Interactive escalations for the agent-facing run/debug tools - #4639

Interactive escalations for the agent-facing run/debug tools#4639
georgi merged 2 commits into
mainfrom
claude/workflow-execution-debugging-n9uea9

georgi commented Aug 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

georgi commented Aug 2, 2026

What

How

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants