Workflow Supervisor PRs 4 and 5: the CLI surface and workflows as agents

[georgi]

Ships the two parallel PRs of Phase B from docs/workflow-supervisor-implementation-plan.md. Both depend on PR 3 (#4634) and on nothing else, so they land together.

PR 4 — the CLI surface

• ExecutionSessionOptions.supervisor in packages/execution is the single integration point (design §7). It forwards to WorkflowRunnerOptions.supervisor; no CLI code touches WorkflowRunner, and no grandfathered direct-construction site gained supervision.
• Flags --supervise, --max-decisions, --max-retries, --supervisor-cost-cap, --supervisor-model on nodetool run, nodetool workflows run, and nodetool debug. Parsing and model-spec resolution are pure (packages/cli/src/supervisor.ts); the agents package and the provider load behind dynamic imports, so an unsupervised run pays nothing. debug goes through the shared debug service — the collector folds supervisor_decision into ExecutionSummary.interventions, and intervention warnings feed the verdict's warnings, never its issues.
• Output: inline ⛨ lines streamed from the session messages (on stderr, so --json stdout stays parseable), a supervised summary line (⛨ supervised: 2 skipped, 1 retried, 3 decisions, +$0.0200), and an interventions block in --json. The record is protocol's Intervention unchanged, so PR 6's UI consumes one shape.
• Cost: recordSupervisorCost writes one Prediction row per billable decision — node_type: "supervisor", workflow_id = the run, metadata.job_id/decided_by/verdict. Every nodetool costs subcommand sees it.

Two deviations, both recorded in the plan: nodetool run is a DSL command executing through @nodetool-ai/dsl's own runner, so --supervise switches it onto ExecutionSession while the unsupervised path stays untouched; and the plan's "198/200 items" is a batch the kernel never names, so the summary takes an optional item total and otherwise reports decisions.

PR 5 — workflows as agents

• The fourth branch. AgentOptions.graph?: GraphData | { workflowId: string } plus supervise, supervisorBounds, maxSupervisorCostUsd. Agent._executeImpl dispatches to executeSuppliedGraph before every planning mode; the mechanics live in packages/agents/src/workflow-agent.ts — child context with shared memory, messages forwarded to the parent, signal wired to session.cancel, getResults() returning the run outputs. Supervision builds BoundedHandle(SupervisorAgent) from the agent's own provider and reasoning model. The branch adopts the existing AgentPolicy: maxStepIterations/maxTokens reach the graph's Agent nodes through the same applyRunPolicy the graph-planner branch uses, so there is no fifth ad-hoc policy.
• Websocket. RunJobRequest.supervise and supervisor?: SupervisorRunOptions. createRunSupervisor is the single factory shared by the main run path and the headless/trigger runner; it returns null unless supervise === true and a provider and model resolve (request → connection defaults → env). supervisor_* messages pass through the relay untouched, now under test.
• Triggers carry a supervise column (sqlite + pg, migration 20260801_000001), defaulting to 0 and forwarded by the dispatcher; registration sync mutates rows in place so a re-sync never resets it. Nothing is flipped on anywhere — the default flip belongs to PR 8's gate.

Merge reconciliation

Both branches independently added ExecutionSessionOptions.supervisor (PR 5 cannot run without it). Resolved to a single field with the stricter doc comment, and to the conditional-spread forwarding form in session.ts.

Tests

New: packages/execution/tests/supervisor.test.ts, packages/cli/tests/supervisor.test.ts, packages/agents/tests/agent-graph-mode.test.ts, packages/execution/tests/session-supervisor.test.ts, packages/websocket/tests/run-supervisor.test.ts, packages/websocket/tests/supervisor-relay.test.ts.

The graph-mode suite covers the load-bearing claims: outputs identical to a bare WorkflowRunner on a clean graph, a scripted skip completing a run that would otherwise fail while surfacing supervisor_escalation/supervisor_decision, a scripted fail still failing, a clean supervised run emitting no supervisor_* at all, and an aborted signal cancelling the run.

Verification

Run on the merged tree, not on the individual branches:

• npm run build:packages — 60/60
• npx vitest run --root packages/{execution,agents,cli,websocket,kernel,models} — 23, 1877, 458, 2008, 1001, 783 passed
• npm run lint — exit 0 (pre-existing react-hooks warnings only, none in touched files); check:deps, check:circular, check:execution-boundary, check:coupled-deps, check:lockfile all pass
• npm run typecheck — web and electron clean. Mobile fails for an environmental reason: mobile/ is not a root workspace and its node_modules was never installed in this container, so every error is TS2307: Cannot find module 'react-native' / 'expo-*' plus TS6053: File 'expo/tsconfig.base' not found. No error names a file in this diff, and no mobile file was touched.

Out of scope

No UI (PR 6), no Assert node (PR 7), no ReplayHandle or eval suite (PR 8), no default flipped on. There is no editor or API surface for setting a trigger's supervise bit yet — the column, model field, and dispatcher read exist; the write path is a PR 6/8 toggle. The browser debug surface stays unsupervised until PR 6.

---

Generated by Claude Code

[Copilot]

Pull request overview

Adds opt-in “workflow supervision” across NodeTool’s CLI and WebSocket run surfaces, and introduces an agent execution mode that runs an existing workflow graph directly (with optional supervision), while keeping the integration point centralized in ExecutionSessionOptions.supervisor.

Changes:

• Introduces ExecutionSessionOptions.supervisor plus shared intervention reporting/rollups (⛨ lines, summaries, warnings) in @nodetool-ai/execution.
• Adds CLI flags (--supervise + bounds), supervised DSL runs, cost ledger attribution for supervisor decisions, and debug output wiring.
• Extends run APIs (protocol + websocket + triggers) to support supervised runs and adds “workflows as agents” (Agent({ graph })) execution path.

Reviewed changes

Copilot reviewed 42 out of 43 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/websocket/tests/supervisor-relay.test.ts	Verifies supervisor_* messages are relayed to WS clients with job/workflow IDs.
packages/websocket/tests/run-supervisor.test.ts	Tests createRunSupervisor opt-in behavior and fallback selection.
packages/websocket/src/unified-websocket-runner.ts	Wires per-request supervision into websocket runs via createRunSupervisor.
packages/websocket/src/triggers/dispatcher.ts	Forwards per-trigger supervise bit into headless trigger runs.
packages/websocket/src/run-supervisor.ts	Adds server-side supervisor factory (provider/model resolution + bounded handle).
packages/websocket/src/headless-job-runner.ts	Adds supervised option to headless job runner and forwards handle into ExecutionSession.
packages/protocol/src/api-types.ts	Extends RunJobRequest with supervise + supervisor options; defines SupervisorRunOptions.
packages/models/tests/migrations.test.ts	Updates expected migration count after adding trigger supervision column.
packages/models/src/trigger-registration.ts	Adds supervise field to trigger registration model + defaulting.
packages/models/src/schema/trigger-registrations.ts	Adds supervise column to sqlite trigger registrations schema.
packages/models/src/schema-pg/trigger-registrations.ts	Adds supervise column to postgres trigger registrations schema.
packages/models/src/migrations/versions.ts	Adds migration 20260801_000001 to add trigger_registrations.supervise.
packages/models/src/db.ts	Updates sqlite schema SQL to include trigger_registrations.supervise.
packages/execution/tests/supervisor.test.ts	Tests session supervision behavior + intervention formatting/rollups.
packages/execution/tests/session-supervisor.test.ts	Tests facade forwarding of supervisor handle (skip rescues, none escalates without handle).
packages/execution/src/types.ts	Adds ExecutionSessionOptions.supervisor and re-exports SupervisorHandle.
packages/execution/src/supervisor.ts	Implements shared intervention rollups and printable ⛨ lines.
packages/execution/src/session.ts	Conditionally forwards supervisor handle into WorkflowRunner options.
packages/execution/src/index.ts	Re-exports intervention reporting + adds collectInterventionWarnings.
packages/execution/src/debug/verdict.ts	Adds intervention-derived warnings (not issues) for supervised runs.
packages/execution/src/debug/types.ts	Adds interventions + intervention counts to ExecutionSummary.
packages/execution/src/debug/index.ts	Re-exports intervention reporting from debug entrypoint for hosts.
packages/execution/src/debug/collector.ts	Folds supervisor_decision into ExecutionSummary.interventions.
packages/cli/tests/supervisor.test.ts	Tests CLI supervisor flag parsing, model-spec parsing, and cost attribution.
packages/cli/src/supervisor.ts	Implements CLI supervision flags, dynamic handle creation, streaming ⛨ lines, and cost ledger writes.
packages/cli/src/run-dsl.ts	Refactors DSL workflow loading and adds DSL→graph mapping for supervised path.
packages/cli/src/run-dsl-supervised.ts	Adds nodetool run --supervise implementation via ExecutionSession.
packages/cli/src/nodetool.ts	Wires supervision flags into run and workflows run, including json shape and cost ledger.
packages/cli/src/debug/verdict.ts	Incorporates supervision warnings into debug verdict.
packages/cli/src/debug/types.ts	Adds supervised metadata to debug report options and server report types.
packages/cli/src/debug/server-runner.ts	Adds supervisor handle wiring + ⛨ streaming + supervised rollup in server debug report.
packages/cli/src/debug/markdown.ts	Renders interventions section into debug bundle markdown.
packages/cli/src/debug/harness.ts	Plumbs supervision options and intervention-line sink into debug harness.
packages/cli/src/commands/debug.ts	Adds supervisor flags to nodetool debug and prints interventions/warnings in summaries.
packages/agents/tests/agent-graph-mode.test.ts	Adds coverage for Agent({ graph }) execution and supervised outcomes.
packages/agents/src/workflow-agent.ts	Implements workflow-as-agent runner (execute graph via ExecutionSession, forward messages, cancellation).
packages/agents/src/index.ts	Exposes workflow-agent utilities/types from package entrypoint.
packages/agents/src/agent.ts	Adds graph execution branch to Agent, with optional supervision via SupervisorAgent.
packages/agents/package.json	Adds dependency on @nodetool-ai/execution for graph-run branch.
package-lock.json	Lockfile update for new agents→execution dependency.
docs/workflow-supervisor-implementation-plan.md	Marks PR4/PR5 as shipped and documents deviations/added mechanics.
docs/cli.md	Documents supervised run flags/output/cost behavior.
CLAUDE.md	Documents supervised runs and nodetool debug --supervise usage.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/websocket/tests/supervisor-relay.test.ts:97

• This test uses a fixed setTimeout sleep to wait for async relay output, which is prone to flakiness and is discouraged by docs/DEVELOPMENT_STANDARDS.md (Testing §8: async assertions should use waitFor-style polling, not arbitrary sleeps). Prefer polling until the expected message arrives.