AGENTS.md

Agents Package

Agent Memory (@nodetool-ai/runtime → context.memory)

Every ProcessingContext carries an AgentMemory instance at context.memory. It is the single source of truth for everything shared between steps, tasks, sub-agents, and tools. Do not introduce a parallel result map in any executor — read and write through context.memory.

Access pattern: progressive disclosure via tools

Memory contents are NOT auto-injected into prompts. Agents access memory through three capabilities that are auto-attached to every step (and to every team iteration). They are the shared capability module (src/capabilities/shared.ts) — kept apart from memory, which is the user's durable store and a different lifetime — and getSharedTools() (src/tools/shared-tools.ts) is the belt the executors mount them from:

Tool	Object model	Purpose
list_shared	nodetool.shared.list(filters)	Discover available entries (metadata only — keys, titles, kinds, byte sizes)
read_shared	nodetool.shared.read(keys)	Fetch full values for specific keys
share_result	nodetool.shared.publish(key, value, {title, description})	Publish a value under shared:<key>

The default execution system prompt documents the nodetool.shared form, and the three wire names drop out of the raw tool catalog the way every other wrapped tool does. The user message names only specific upstream keys the planner pinned (step.dependsOn plus parent-task dependsOn via upstreamMemoryKeys) — values are pulled on demand.

Key namespaces

import { memoryKeys } from "@nodetool-ai/runtime";

memoryKeys.step("step_1");         // "step:step_1"  — step result
memoryKeys.task("research_phase"); // "task:research_phase"  — task result
memoryKeys.input("customer");      // "input:customer"  — caller-supplied input
memoryKeys.shared("note");         // "shared:note"  — cross-agent scratch

Who writes what

Writer	Trigger	Key	Kind
CodeActExecutor	Step completion	step:<step.id>	step_result
CodeActExecutor	Last step of a task (finish-task)	task:<task.id>	task_result
TaskExecutor	Startup / terminal step failure	input:<key> / step:<step.id>	input / step_result
ParallelTaskExecutor	After a task completes (idempotent)	task:<task.id>	task_result
share_result tool	Agent / sub-agent publish	shared:<key>	shared

share_result is restricted to the shared: namespace so agents can't spoof step / task / input results. Internal executors write directly through context.memory.set for their owned namespaces.

Custom prompts are preambles, not replacements

A step executor always builds the default execution prompt (the CodeAct action contract, the output schema, the finish() discipline). A caller-supplied systemPrompt is layered as a preamble before the default — it cannot override the execution contract. Earlier versions allowed this and broke result capture in plan mode.

TaskPlanner follows the same rule, and did not until recently: a caller's systemPrompt replaced the whole TaskArchitect contract — ID rules, parallelism, output schemas, "synthesis is not your job". Every AgentNode supplies one (default: "You are a friendly assistant"), so every node-driven plan was authored without it. It is a preamble now.

The contract's tool-dependent half is conditional on the run's toolbelt. With tools it carries the find_model / generate_* / save_asset instructions; with an empty belt it carries ## No Execution Tools instead, and the planner is told to plan only what a model can do from its own knowledge. Advertising tools no step can call produced research plans whose every step was the model's own recall.

Final synthesis: the calling loop's next turn

There is no synthesis stage. execute_plan runs the DAG, writes each task's result to task:<id>, and returns those values with the tasks; the loop that called it then writes the answer on its next turn, reading task:<id> via read_shared for anything the return did not carry. That is what execute_plan's own description tells the model to do.

The planner is still told NOT to create an aggregation/synthesis task: a step that assembles the answer duplicates the turn that follows the call, and it does it with a smaller view of the run than the caller has.

Threading task-level deps through executors

ParallelTaskExecutor derives task.dependsOn.map(memoryKeys.task) and forwards it as upstreamMemoryKeys to TaskExecutor, which forwards it verbatim to every step executor. The step's user message renders these as - task:<id> hints next to the intra-task step:<id> deps. The agent calls read_shared when it needs the values.

Tests

• packages/runtime/tests/agent-memory.test.ts — unit tests for AgentMemory
• packages/agents/tests/shared-tools.test.ts — unit tests for the shared capabilities list_shared / read_shared / share_result, and the belt getSharedTools() builds from them
• packages/agents/tests/memory-propagation.test.ts — end-to-end through execute_plan, TaskExecutor and StepExecutor, including a fake-provider round trip that drives list_shared → read_shared → finish_step
• packages/agents/tests/_helpers/mock-context.ts — shared mock context with a real AgentMemory for executor tests

When asserting memory writes in tests, prefer context.memory.has(memoryKeys.task("...")) and context.memory.subscribe(...) over spies on set / storeStepResult.

For the full API reference, tool schemas, propagation flow, design decisions, and troubleshooting, see docs/agent-memory.md.

JavaScript Sandbox (src/js-sandbox.ts)

Reader-facing reference for this whole section: docs/javascript-sandbox.md.

User-authored JS from a CodeAct action and nodetool.code.Code runs in a QuickJS WebAssembly sandbox via @sebastianwessel/quickjs. The guest lives in its own WASM heap, so runaway or malicious code can't corrupt the host V8 heap the way it could under the previous node:vm implementation.

Windows test portability: filesystem-containment tests must use tests/_helpers/filesystem-links.ts instead of calling symlink() directly. The helper creates directory junctions on Windows, so the tests do not require Developer Mode or elevation, and regular directory symlinks elsewhere. Normalize paths before comparing them with repository-style / paths, and use Node itself for cross-platform process-spawn tests instead of POSIX utilities such as true. Normalize wrapper-produced module IDs before matching them: QuickJS uses \ on Windows and / on POSIX. Keep native media fixtures on software encoders; a Windows hardware probe can find a codec that the installed driver cannot open.

On Node the interpreter runs on a worker_threads worker (src/js-sandbox-worker/): buildSandbox and every host bridge stay on the main thread, the worker rebuilds each bridge as an RPC proxy over the port, and a CPU-bound guest therefore blocks only its own thread — the server keeps serving, and the websocket stop frame that cancels the run can still be read. Abort is worker.terminate(), immediately — equally instant for a spinning guest and a parked one, and nothing of value dies with the thread: logs and emitted values accumulate main-side through the RPC dispatches, and a cancelled action leaving no partial state write-back is the cleaner contract. Three runs stay in-process: the browser (no worker_threads), a run with input streams (the synchronous stream.open mirror has no handle names to seed), and NODETOOL_SANDBOX_INPROC=1. NODETOOL_SANDBOX_WORKER=require turns any other fallback into an error for CI. Under tsx/vitest the worker boots from an eval'd bootstrap that registers tsx/esm/api — tsx's own --import hook skips worker threads, and Node's built-in strip-only TS loader cannot parse the workspace packages. The packaged backend ships the worker as a second esbuild bundle (scripts/bundle-backend.mjs → backend/js-sandbox-worker/worker-entry.js). The interpreter itself (js-sandbox-worker/interpreter.ts) is shared by both paths, so they cannot drift; bridge results must be plain data — a function in a bridge return value cannot cross postMessage, which is why the fetch bridge returns body bytes and the guest prelude builds text()/bytes()/ arrayBuffer() over them.

Hard limits enforced by the runtime. Each row's default can be overridden per invocation via RunSandboxOptions.limits, clamped to the ceiling in the last column by resolveSandboxLimits:

Limit	Default	Configured by	Ceiling
Execution time	timeoutMs (30 s)	setInterruptHandler (CPU budget) + wall-clock race	—
Suspended time	DEFAULT_SUSPEND_ALLOWANCE_MS (30 min), or INPUT_STREAM_SUSPEND_ALLOWANCE_MS (unbounded) for a run with onTakeInput	RunSandboxOptions.suspendAllowanceMs, only with a clock	—
Guest heap	GUEST_MEMORY_LIMIT = 64 MB	runtime.setMemoryLimit (limits.memoryLimitBytes)	512 MB
Call stack	GUEST_STACK_LIMIT = 512 KB	runtime.setMaxStackSize (limits.stackLimitBytes)	8 MB
Fetch calls	MAX_FETCH_CALLS = 20 per run	counter inside bridge (limits.maxFetchCalls)	100
Fetch body	MAX_RESPONSE_BODY_SIZE = 1 MB	truncation inside bridge (limits.maxResponseBodyBytes)	50 MB
Fetch timeout	FETCH_TIMEOUT_MS = 15 s	per-request AbortController (limits.fetchTimeoutMs)	120 s
Output	MAX_OUTPUT_SIZE = 100 KB	serializeResult truncation (limits.maxOutputSize)	10 MB
Random bytes	MAX_RANDOM_BYTES = 64 KB	crypto.getRandomValues clamp	—
Progress reports	MAX_PROGRESS_CALLS = 1000 per run, one per PROGRESS_MIN_INTERVAL_MS = 100 ms	counter + timestamp inside the bridge	—
Host module text input	MAX_HOST_INPUT_CHARS = 5 MB	check inside host-modules/limits.ts	—
Host module byte input	MAX_HOST_INPUT_BYTES = 10 MB	check inside host-modules/limits.ts	—
sandbox-html matches	DEFAULT_SELECT_HTML_LIMIT = 100	options.limit	MAX_SELECT_HTML_LIMIT = 1000
sandbox-zip inflation	MAX_UNZIP_TOTAL_BYTES = 50 MB total	check inside host-modules/zip.ts	—
sandbox-xlsx write	MAX_WRITE_SHEETS = 64, MAX_WRITE_CELLS = 250 000	checks inside host-modules/xlsx.ts	—
sandbox-ocr words	MAX_OCR_WORDS = 20 000	check inside host-modules/ocr.ts	—
image.* input	MAX_IMAGE_INPUT_BYTES = 25 MB	length check inside each bridge	—
Image / canvas pixels	MAX_IMAGE_PIXELS = 32 M, longest edge MAX_IMAGE_DIMENSION = 16384	assertSurfaceSize	—
image.decode pixels	MAX_DECODE_PIXELS = 8 M	check inside the bridge	—
Canvas draw ops	MAX_CANVAS_OPS = 10 000 per render	count inside renderCanvas	—

QuickJS's memory limiter counts its own heap objects; string and typed-array payloads are not charged against it, so memoryLimitBytes bites on object allocation, not on new Uint8Array(n).

Secrets are scoped. SandboxLimits.secretScope (host-set, null for an unscoped run) is the list of names a run may read; the getSecret bridge refuses everything else, and nodetool.secrets.get/tryGet/list sit on top of that one check rather than beside it. nodetool.code.Code exposes the list as its secrets property, empty by default so a node written before scopes existed keeps the reach it had. Tests: tests/sandbox-secrets.test.ts.

Exposed guest surface: console, fetch, sleep, getSecret, crypto.{randomUUID,getRandomValues,digest,hmac} (WebCrypto-backed — digest and hmac take SHA-1/256/384/512 and accept string or Uint8Array input, both returning a Uint8Array), workspace.{read,write,list,readBytes,writeBytes, stat,root,copy,move,mkdir,remove} (requires a ProcessingContext carrying a Workspace; remove deletes one file or one empty directory, never a tree; stat returns {exists, size, isDirectory, isFile, isSymlink, modifiedMs, createdMs, accessedMs} and reports a missing path as exists: false rather than throwing). Those calls go through context.workspace, not node:fs, so a Code node behaves the same whether the run's workspace is a folder or a prefix in object storage; root() answers the real directory when there is one and /workspace otherwise, and containment is the workspace's own rule rather than a symlink check here. filesystemAccess: "host" — the lib.os escape hatch a host opts into, never guest code — still resolves real paths through node:fs, the pure guest-side helpers toBase64/fromBase64/toHex/fromHex/parallelMap/createCanvas (UUIDs come from crypto.randomUUID and UTF-8 from the native TextEncoder/TextDecoder — the old uuid/utf8Encode/utf8Decode aliases are gone), progress(percent, message?), format.{number,date,relativeTime,list}, image.{info,stats,decode,blank,pad,grid,resize,crop,rotate,flip,adjust,composite,convert}, audio.{info,normalize,trim,concat,mix,reverse,fadeIn,fadeOut,repeat}, video.{info,trim,resize,rotate,addAudio,extractAudio,extractFrame}, canvas.measureText (plus canvas.render, the undocumented plumbing behind createCanvas(...).toBytes()), and any caller-supplied globals. fetch sends a Uint8Array body as raw bytes instead of JSON. Every one of these is a capability, not a library — libraries are imports (below).

stream is the input side of emit, built in the guest prelude over one awaitable host bridge: for await (const item of stream(name)) reads one input handle in order until end-of-stream, stream.any() interleaves every handle as [handle, value] pairs, await stream.first(name) takes the next value (undefined at end-of-stream), and stream.open(name) answers synchronously whether more can arrive. The host supplies the values through RunSandboxOptions.onTakeInput (called with null for any()) and the probe through onStreamOpen; without an onTakeInput every verb throws NO_INPUT_STREAM_MESSAGE. The guest pulls, so an item nobody asked for stays in the host's inbox — backpressure costs nothing. Time parked on a take is clock-suspended: a streaming body's timeout meters its own execution, and how long upstream takes is the run's cancellation to bound, not the timeout's. Tests: tests/js-sandbox-stream.test.ts.

progress is fire-and-forget: it reports to RunSandboxOptions.onProgress, clamped to 0–100 with the message truncated to 500 chars, and is a no-op when the caller passes no sink. nodetool.code.Code wires it to context.postMessage({ type: "node_progress", … }), the same channel the Python worker uses, so a long-running snippet drives the node's progress bar.

image, audio, video, and canvas are the media namespaces. Image and canvas are host bridges over a real 2D canvas (src/sandbox-media.ts). The backend is picked at first use: @napi-rs/canvas (Skia) on Node, loaded through importHidden so no bundler pulls the native addon into a browser graph — it is already staged as an external by scripts/bundle-backend.mjs — and OffscreenCanvas + createImageBitmap in the browser runner. Ops: info, stats, decode, blank, pad, grid, resize (fit: cover/contain/fill), crop, rotate (grows to the rotated bounding box), flip, adjust (the CSS filter set), composite (layers with position, size, opacity and globalCompositeOperation blend mode) and convert. Encoding to jpeg fills transparency with background, white by default.

image.encode is gone. It took width*height*4 bytes from the guest, so using it meant building a pixel array in the guest heap and shipping it across — and every observed use was a backdrop, 6 MB of zeros moved to produce a 6 KB PNG. After handles it was the largest guest→host payload left in a normal image run, and the manifest advertised it, so it read as the way to make an image. Three ops cover what it was reached for, all host-side with nothing crossing: blank(width, height, {color}) for a surface, pad(image, {all|top|right| bottom|left, color}) to grow the canvas without scaling (what nodetool.image.CanvasResize does in a graph), and grid([image, …], {columns, gap}) to lay images out (which retired lib.grid.CombineImageGrid) — which is the whole of "generate two images and combine them", the task that started this. Host code that genuinely holds pixels keeps the direct path through encodePixels, which is not on the guest bridge.

image.stats(image) is the counterpart to handles: once the guest stops holding images it cannot answer anything about one, and decode — the only way to look — pulls every pixel across, which is the cost handles removed. stats reads them host-side and answers in a hundred bytes: per-channel mean/min/max, mean luminance, and whether the image is opaque. It carries no MAX_DECODE_PIXELS bound because it transfers nothing.

Media transforms trade in handles, not bytes (src/sandbox-media-handle.ts). Each image, audio, or video op takes a handle, a media ref (asset:// and every uri media.* resolves), or raw bytes, and returns a handle: a small plain object ({uri: "sandbox://media/<id>", mimeType, byteLength, width, height}) naming bytes the host holds for the run. So a chain — and a generated image fed straight in — moves nothing across the boundary but small objects.

That is the difference between working and not. The guest is a courier: in the shape that motivated this (generate two images, combine them) it never reads a pixel. Carrying them as bytes cost a guest→host copy plus a base64 round trip per hop, several times the payload live in the guest, and past roughly 16 MB a run aborted the runtime at teardown. A long op chain over a single photo-sized PNG went from aborting after a few ops to finishing in seconds once handles replaced bytes.

Audio and video transforms use Mediabunny directly. Browsers use WebCodecs; Node registers Mediabunny's server codec adapter. No audio-node or video-node package is imported or exposed to guest code. video.extractAudio returns an audio handle and video.extractFrame returns an image handle.

Two image exceptions, both returning plain data: info and decode exist to report what is in an image, and decode is the one call that must hand over real pixels. image.bytes(handle) is the only door back to encoded bytes and needs no context — reach for it when the body parses the bytes itself, not to move an image between calls. media.toImage(handle) promotes a handle to a durable asset; nothing else writes storage, so a three-op chain does not leave three rows behind. A handle lives for its run: using a stale one says so rather than blaming its uri. limits.runMediaBytes (default 256 MB) bounds what one run may hold — the aggregate the per-call ceilings never covered, so exhaustion is a sentence naming the limit instead of an Emscripten assertion.

A canvas context is a host object with methods, which the plain-data bridge contract cannot carry, so drawing is recorded rather than proxied: the guest helper createCanvas(width, height) returns a surface whose getContext("2d") takes the ordinary Canvas 2D calls synchronously, appending each to a draw list, and await surface.toImage({format, quality, background}) ships the whole list through canvas.render to be replayed against a real context and encoded, answering with a handle like every other image producer. toBytes() keeps its name's promise and pays for the bytes explicitly. drawImage takes image bytes, not an image object. Gradients work the same way — createLinearGradient returns a tagged handle that the renderer swaps for the real gradient when it is assigned to fillStyle. The method and property allowlists live in src/sandbox-canvas-api.ts, read by both the guest recorder and the host replay, and an op naming anything outside them is refused. The recorded ops are marshaled out of the guest one object at a time (~1 ms each), which is what MAX_CANVAS_OPS really bounds — for heavy composition reach for image.composite instead of tens of thousands of primitives. canvas.measureText(text, font?) returns text metrics so text can be laid out before it is drawn.

Libraries are imports, never globals. Every library the sandbox offers is a sandbox package a node imports at the top of its body — the import is the declaration; there is no data.* namespace any more. Two kinds:

• Guest packs — the M3 compiler bundles the library into QuickJS: @nodetool-ai/sandbox-yaml (js-yaml), -dates (date-fns), -markdown (marked).
• Host packs (src/host-modules/) — the library runs where the sandbox runs, behind a generated ESM facade over a per-run dispatcher: -csv (papaparse), -html (cheerio + turndown), -xml (fast-xml-parser), -xlsx (exceljs), -zip (fflate), -diff (diff), -ocr (tesseract.js), -tfjs (TensorFlow.js and its model zoo), -docx (docx), -mammoth (mammoth), -epub (epub2), -pptx (office-text-extractor), -tokens (js-tiktoken). These are the libraries the guest cannot hold — Node builtins, a DOM, a file path instead of a buffer, a limit the guest could not enforce on itself, or state that has to outlive a run. Four more are NodeTool's own code rather than a library: -aws (SigV4 signing), -notion, -supabase and -twilio build an authenticated request and return it; the guest sends it with its own fetch, so the fetch cap and the SSRF guard still apply. They replace the S3/Notion/Supabase/Twilio nodes. Apify was a fifth and is not: handing the guest a token to fetch and poll with is the wrong shape for a paid service running third-party code, so it became the apify capability module instead (docs/apify-integration.md).

Host modules (src/host-modules/)

The host-JS analog of the WASM path below, and the same mechanism. A pack's manifest entry is {"kind": "host", "host": "<id>"} — an id, never code. SANDBOX_HOST_MODULES (@nodetool-ai/protocol) is the registry: it names every id, the one package allowed to declare it, and its exports. The specifier resolves to generateSandboxHostFacade's output — one async export per registry export plus a default namespace — importing the private nodetool:host-bridge module, which the loader serves only to generated facades.

createSandboxHostDispatcher is the boundary. It refuses a resolution naming an unknown id or claiming another pack's id, then validates the module key, the export name and the argument list on every call before an implementation is even loaded. The dispatcher binding is deleted before the user IIFE starts; a module that grabs it during linking gains nothing beyond the run's declared surface.

registry.ts loads each implementation lazily, and each implementation imports its library lazily inside itself — so nothing sits in an entry graph, esbuild still inlines them into the packaged server.mjs, and Vite resolves the browser builds for the in-browser runner, where the "host" is the page. Results go out as plain data with bytes tagged at any depth (toGuestBytesDeep, sandbox-bytes.ts), and errors as tagged objects — the marshaling rule every bridge follows.

Safety limits live inside the implementations, where nothing can route around them: MAX_UNZIP_TOTAL_BYTES in zip.ts, the select limits in html.ts, the write caps in xlsx.ts, the shared input caps in limits.ts. Tests: tests/host-modules.test.ts (libraries end to end, the dispatcher's refusals, a forged manifest, every limit) and packages/sandbox-compiler/tests/packs.test.ts (every shipped pack through the real install path).

The guest's own loader serves only the sandbox packages a run may reach — for a Code node, the packs its body statically imports, resolved against the host's catalog; for a CodeAct session, its allowlist. Dynamic import() and require never resolve. The browser runner fetches those modules over GET /api/sandbox-modules/* and verifies each body before it runs, so the same rules hold client-side.

A session that allows packages also carries get_sandbox_package_docs (capabilities/packs.ts): it serves one pack's SKILL.md, refuses a specifier off the session allowlist, and wraps the body of a pack the operator has not put on the pack-loader allowlist in <untrusted-package-docs> — read as reference, never as instructions. The ambient prompt tier stays one sanitized, capped line per allowed specifier.

sandboxPackageSkills (codeact/sandbox-package-docs.ts) turns a trusted pack's skill into an ordinary AgentSkill, which is the injection tier the untrusted path is contrasted against. No host mounts it. Its one caller was the retired Agent, so a trusted pack's body reaches the model only through get_sandbox_package_docs. Wiring it into the chat session's skill catalog is the follow-up; nothing depends on it being absent.

Host WASM modules (src/wasm-sandbox/)

A pack may declare a WASM module. Its specifier resolves to a generated ESM facade (generateSandboxWasmFacade, @nodetool-ai/protocol) with one async export per manifest export, calling a per-run dispatcher through a private bridge module. The call contract is scalar-only: i32/f32/f64, at most 8 arguments, at most one result, and a void export resolves undefined.

Stateless by contract. Each call instantiates fresh from the cached module inside the worker, runs, and discards the instance — mutable globals and linear memory never carry from one call to the next. A pack needing state keeps it in guest JS and passes scalars in.

Bound	Default (also the ceiling a manifest may lower to)
Worker pool (process-wide)	4 — not manifest-configurable
Call concurrency per invocation	2
Calls per invocation	256
Aggregate WASM wall clock per invocation	30 s
Per-call timeout	5 s, then the worker is terminated and replaced

The aggregate wall clock is a reservation, not a meter read on completion. A call is admitted only after taking min(perCallTimeout, remaining) out of the budget, so concurrent calls divide one cap instead of each seeing the whole remainder, and the reservation is what bounds that call's timeout. On completion the reservation is replaced by the real duration — a fast call refunds, one cut at its timeout keeps the charge — so the budget stays the sum of call durations it documents.

Cancelling the run reaches the worker, not just the caller. The signal is rechecked after every await on the way to dispatch — compiling a module and waiting for a concurrency slot both outlive an abort — and it is passed into the pool, where it terminates the worker the way a timeout does. A guest export has no yield point, so abandoning the promise would leave the thread spinning for the rest of its timeout on work nobody wants.

The dispatcher is the boundary, not the hiding: it serves only the run's declared WASM modules and validates module identity, export allowlist, argument count, and argument type before any worker runs — i32 rejects out of range rather than wrapping, f32/f64 take NaN and infinities. The bridge module is refused to every importer but a generated facade, and the dispatcher binding is deleted before the user IIFE starts. A pack module that grabs the binding during linking gains nothing beyond the run's own declared surface.

workers.ts is the platform seam: Node worker_threads, browser Web Worker, both created from an inline source string so no entry file has to survive tsx, vitest, dist/, and the esbuild backend bundle. The browser path is written but unexercised here — no browser harness lands until M2.

Fixtures: tests/fixtures/sandbox-wasm/ (the reference module, its WAT, and the contract cases as data). Tests: tests/js-sandbox-wasm.test.ts (end to end, real workers), tests/wasm-sandbox-host.test.ts (conversion, budgets, pool).

format exists because QuickJS ships no Intl: each member is a host bridge over Intl.NumberFormat, Intl.DateTimeFormat, Intl.RelativeTimeFormat and Intl.ListFormat, defaulting to locale en-US. All four are async (they follow the never-reject convention), so a bad locale or option arrives in the guest as a thrown Error carrying Intl's own message. eval and Function are deleted at init so the user cannot re-enter dynamic code generation. Core JS (JSON, Math, Date, Map, URL, TextEncoder, etc.) is QuickJS's native implementation, not a host-bridged version.

Async concurrency: a bridge call starts its host-side work when invoked, not when awaited, so Promise.all/allSettled/race/any over fetch, workspace.* or any other bridge run the host operations in parallel — five fetches under Promise.all take one round trip. parallelMap(items, fn, concurrency?) is the bounded form (order-preserving, default 5, max 32, rejects on first failure). The per-run fetch cap counts parallel calls the same as serial ones. Timer globals (setTimeout, setInterval, setImmediate and their clears) are deleted inside the user-code module — the engine re-installs host-backed versions on every evaluation, so the prelude alone can't remove them; wrapCode does. sleep stays the only timer. Tests pinning all of this: tests/js-sandbox.test.ts ("async concurrency").

State sync-back: object-typed globals are deep-replaced on the host after the guest runs, so CodeNode's state object persists across invocations. Primitive globals pass by value (no sync).

Suspending the clock (createSandboxClock, RunSandboxOptions.clock): the timeout bounds guest execution, and a program waiting on a permission prompt is not executing — it is waiting on a person. Charged to the same budget, the wait kills the program that asked, and the answer then resolves nothing. A caller that owns such a wait wraps it in clock.suspend(); the suspended time is added back to timeoutMs, so the program resumes with the budget it had. Suspensions nest, and the engine's own abort moves out to timeoutMs + suspendAllowanceMs as the backstop for a prompt nobody answers. The interrupt handler still cuts a runaway loop at exactly timeoutMs of running time. The websocket chat runner owns one clock per turn and suspends it around every tool- and plan-approval round trip.

Known QuickJS limitations:

• url.searchParams.set(...) doesn't propagate back to the parent URL. Build the query via URLSearchParams directly.
• Host async functions must never reject — js-sandbox.ts wraps them in a neverReject adapter that returns a tagged error object, which a guest prelude rewraps into a real throw. Working around a known handle leak in @sebastianwessel/quickjs@3.0.1 (tracked as list_empty(&rt->gc_obj_list) assertion on runtime dispose).
• Every Lifetime taken from the engine must be disposed, including the one ctx.getArrayBuffer() returns. The typed-array serializer dropped it, so each Uint8Array crossing guest → host leaked a handle and a run moving ~16 MB tripped the same list_empty(&rt->gc_obj_list) assertion — after the guest had computed its answer, so the result was produced and then thrown away. Fixed; pinned by "guest→host binary volume" in tests/js-sandbox.test.ts.
• The wrapper's marshaler recurses on a function: a function's prototype chain leads back to itself, so console.log(fn) — or logging a module namespace, which is what import * as ns gives — blew the host stack, left handles alive, and aborted the runtime on free with the same list_empty(&rt->gc_obj_list). The whole run died, after its work was done, for a debug print. console.* arguments are formatted guest-side now (the init prelude wraps the host console), so only strings cross: a function prints as [Function: name], a cycle as [Circular], and everything else exactly as before. Pinned by "logs a function without killing the runtime" in tests/js-sandbox.test.ts.
• An engine failure never reaches runInSandbox's own await: an Emscripten abort() arrives as a WASM RuntimeError, and a marshaling failure (guest OOM while a host return value is written into the guest) throws inside a promise continuation the library never catches. Unhandled, that killed the host process and the tool call it was serving returned nothing — a chat turn that simply stopped, with nothing for the agent to read or retry. guardHostProcess claims those rejections and fails the run; describeEngineFailure turns them into an actionable message instead of raw assertion text. Anything not attributable to the engine is re-thrown, so a genuine host bug still crashes.
• The wrapper's Node-compat bootstrap is served cheaper sources (src/sandbox-bootstrap-modules.ts). Before our sandboxed function is called it compiles ~12KB of polyfills — node:buffer, node:util, node:url, Headers, Request, Response — into the fresh runtime, and the init prelude then deletes Buffer, Headers, Request and Response. Those imports resolve through our module loader, so four are served as empty modules and node:util as the TextEncoder/TextDecoder pair the guest keeps; node:url still comes from the wrapper, because URL and URLSearchParams are capabilities and a hand-rolled URL parser is not a saving worth making. Measured: 5.96ms → 3.28ms of per-run setup, against 0.34ms for a bare QuickJS context. A wrapper release that renames these modules loses the saving silently, so tests/js-sandbox-modules.test.ts asserts that every id we stub is one the bootstrap actually requests.
• Structured data crosses as JSON, not as handles (src/sandbox-json-transport.ts). The wrapper marshals an object node by node — four evalCode compilations plus a descriptor copy per property — so a run handed 5 000 rows spent ~2 s reading them and ~0.8 s handing them back before running a line of its own. The guest now builds its result with JSON.stringify and the host parses it, the host encodes injected globals for the prelude to parse, and emit/output arguments take the same path; measured 17–35× on those shapes. Typed arrays and strings over 8 KB ride a sidecar the marshaler moves whole, so bytes keep the fast native serializer and a megabyte of text is not escaped and unescaped. Dates, non-finite numbers and undefined properties carry markers. A value the encoder cannot represent (a function, a bigint, a Map, a class instance) falls back to the wrapper's marshaling; a cycle is refused by name, because the fallback follows it until the runtime aborts. Benchmark: npm run bench:sandbox --workspace=packages/agents.
• Binary crosses the boundary asymmetrically. Guest → host is handled by the typed-array serializers (addSerializer), so a guest Uint8Array reaches a bridge as a native one. Host → guest is not: a returned Uint8Array arrives in the guest as a numeric-keyed plain object. Bridges that produce bytes therefore return a base64 marker object and the guest prelude rebuilds a real Uint8Array — the pattern to follow for any new binary bridge.
• serializeResult scans for typed arrays at any depth. It used to look only one level in, so binary nested deeper fell onto the JSON.stringify path, where a Uint8Array becomes {"0":137,"1":80} — lossy, and indistinguishable from a user's own integer-keyed map. The streaming path hit this every time, since genProcess returns an array of yielded objects and the bytes are always at depth 2. The walk is cycle-safe and depth-capped (SERIALIZE_MAX_DEPTH); a cyclic value still falls through to String.

Running Agents from CLI

Interactive Chat

Every session runs the unified agent loop; -a, --agent and --no-agent are accepted for backwards compatibility and do nothing.

# Start a session
nodetool-chat

# With specific provider and model
nodetool-chat --provider anthropic --model claude-sonnet-5

# With workspace directory
nodetool-chat --workspace /path/to/project

# Connect to WebSocket server
nodetool-chat --url ws://localhost:7777/ws

Piped Input

echo "Summarize this codebase" | nodetool-chat --provider anthropic

Interactive Commands

COMMANDS in packages/cli/src/app.tsx is the list; anything else prints Unknown command. There is no /agent: every session runs the unified agent loop.

/help     — Show available commands
/new      — Start a new chat session
/clear    — Clear conversation history
/compact  — Summarize conversation into retained context: /compact [instructions]
/model    — Set model: /model <model-id>
/provider — Set provider: /provider <name>
/tools    — List enabled tools
/exit     — Exit the chat
/quit     — Exit the chat

nodetool agent run

An objective is one CodeAct turn over the same belt chat uses, so there is no second loop to configure. packages/cli/src/commands/agent.ts assembles it: buildCliAgentBelt({ ..., planning: true }) puts create_plan and execute_plan on the belt, createCliCodeActTurn turns that belt into the execute_code action plus the direct tools, and processChat runs the turn. The model decides whether the objective wants decomposing; nothing plans on its behalf.

The stream is the same ProcessingMessage union every host emits — chunk, planning_update, task_update, tool_call_update, tool_result_update, step_result, log_update — which is why --json output and the chat renderer read the same events.

To build a session of your own, use createChatCodeActSession (src/codeact/chat-codeact.ts) rather than assembling an executor: it owns the system prompt, the sandbox package allowlist, the resident/direct tool split, and the clock that stops the action budget while a permission prompt is open.

Capability registry (src/capabilities/)

A capability is what replaced the Tool class: a spec (wire name, description, input schema, and a required PermissionCategory), an impl (run, args) => result, and a CapabilityRun that carries everything only a run knows — the ProcessingContext, the permission gate, the browser router for ui_*, the sub-agent runtime, and the injected singletons (nodeRegistry, providers, examples, exportDsl, loaders). Per-run state arrives at call time, not at construction time, which is what lets one process-level registry serve every host. Design: docs/tool-class-retirement-design.md.

registry.ts is one table, CAPABILITY_MODULES, with one entry per namespace carrying both halves: a lazy loader — one import(), so no implementation sits in an entry graph — and an eager specs list. Every module has a data-only sibling (workflows.specs.ts, media.specs.ts, …) holding wire name, description, JSON schema, category and message template, and importing no implementation, so capabilitySpec(name) and listCapabilitySpecs() answer synchronously. That is what lets a belt be assembled synchronously from the registry: toolFromLazyCapability(spec, run) / toolForCapabilityName(name, run) (capabilities/lazy-tool.ts) return a Tool whose spec is there at assembly time and whose implementation loads from its own module at the first process() — Tool.process() was already async, so only the spec ever had to be eager. getBuiltinTools(), getAllMcpTools() and getGoogleWorkspaceTools() all build this way; the one-line extends CapabilityTool subclasses they replaced are gone, and so is CapabilityTool itself — toolFromCapability(spec, impl, run) (capabilities/adapters.ts) hands the implementation it already has to toolFromLazyCapability's optional impl argument.

A module imports its own specs back and attaches each to an implementation, so one spec object stands behind both halves, and eagerSpecDrift compares them by identity — a module that copied its spec would pass a field check and still be two things to keep in step. CapabilitySpec.zodSchema carries the Zod schema for the capabilities whose identity is one (view_image, list_images, the ui_* document tools, the settings and packs namespaces). It is checked once, by validateCapabilityArgs (capabilities/args.ts): in LazyCapabilityTool.process() on the belt path, in gatedCall on the invoke path. Tool.execute does not pre-parse — LazyCapabilityTool exposes no schema — so neither entrance validates twice and neither skips it.

What the registry deliberately does not serve is written down where the API it mirrors lives: packages/websocket/src/trpc/sandbox-coverage.ts classifies every tRPC procedure as covered by a capability, reachable through a differently-shaped one, withheld with the risk stated, or a recorded gap. packages/websocket/tests/sandbox-api-coverage.test.ts walks the live router and fails in both directions, so a new procedure cannot land unclassified and a stale verdict cannot outlive what it judged. The rule the withheld set follows: a run may act on the rows its own user owns, and may not touch credentials, billing, other tenants, host control, the transcript of its own behaviour, or anything that grants a third party access.

DECLARED_CAPABILITY_MODULES is the module list a reviewer reads, derived from CAPABILITY_MODULES — a declared module with no loader, or a loader nobody declared, cannot occur, because both come from the same entry. Three drift walks keep the rest honest. capabilityModuleDrift() reports an export with no name, description, schema, category or implementation (capabilityModuleIssues), a spec object a module rebuilt instead of importing from its .specs.ts sibling (eagerSpecDrift), and one name owned by two modules; tests/capabilities-registry.test.ts also pins a checked-in name → category snapshot, so a reclassification is a one-line diff. tests/capabilities-coverage.test.ts walks the other way: everything getBuiltinTools() and getAllMcpTools({}) assemble must resolve through findCapability, or sit in that file's pinned exception list with a reason.

invoke.ts holds the one ladder — lookup, gate, impl. Every entrance runs it: the guest dispatcher, a direct MCP registration, run_subtask's child loop. gateTools (capabilities/gate-tools.ts) is the door a Tool walks through: it wraps each tool in a subclass whose process() builds a one-call run over capabilityFromTool and calls invoke. tests/capabilities-gate-parity.test.ts drives both entrances and compares transcripts.

The guest reaches a namespace by import, never by a global:

import { list_workflows } from "@nodetool-ai/sandbox-nodetool/workflows";
const { workflows } = await list_workflows({ limit: 20 });

Exports carry the wire name — list_workflows, not listWorkflows — so the prompt, the MCP surface and the belt all say one string. The facade generator and specifier shape live in @nodetool-ai/protocol (sandbox-capability.ts); the module list stays in the registry here, and the host decides which modules a session mounts. A third-party pack can never declare one. createCapabilityDispatcher validates module key, export name and argument list on every call, then delegates to invoke — it never gates on its own, so the import path and the belt bridge reach one implementation past one gate.

coerceCapabilityArgs (capabilities/args.ts) is the argument half of that check, and it both folds and refuses. It folds camelCase onto snake_case, a lone string onto the first required string field, and a bare id onto the one required *_id key — only when there is exactly one, so nothing is guessed. Then it refuses a call that brought arguments but not the required ones, naming the call, the missing keys and what was passed instead. Before that, get_workflow({ id }) reached the implementation as workflow_id: undefined and answered "Workflow undefined was not found" — a report about a missing workflow for what was a misspelled argument, and one an agent read as a broken database. It cost six calls and three wrong theories in one session. A call with no arguments is still passed through: {} is the documented shape for the capabilities that require nothing, and for the rest the implementation's own "x is required" already names the field. The flat tools.<name>() global is gone: every capability is an import, and what a session added at its own call site is grafted onto .../session (client ui_* tools onto .../ui) so one import shape covers everything. The nodetool object model stays a global.

Import direction is one-way: capabilities/ imports tools/tool-permissions.ts, never the reverse. That is why gateTools sits in capabilities/ and not beside the classification map it uses. The reverse edge made the bundled backend's module wrappers an async cycle — init_tool_permissions awaiting init_adapters awaiting init_tool_permissions — which esbuild's __esm cannot break the way real ESM breaks a synchronous cycle, and server.mjs died on an unsettled top-level await before it served /health. npm run backend:smoke is the check that catches it; a passing vitest run will not.

Capability coverage — every capability names its check

SurfaceEntry.paths in the harness registry is coarse on purpose: packages/agents/ is one path, so a diff that adds a capability lights up exactly the checks a diff that renames a local variable does. Five capabilities shipped through that hole before anyone noticed nothing exercised them (#5095, #5100, #5103, #5105, #5107).

So there is a second table one rung down: packages/cli/src/harness/capability-table.ts. One entry per exported capability, naming the file that implements it, the checked-in suites a selfcheck runs over it, the eval cases whose expect.requiredTools demand it, and — where nothing does yet — a written gap note. The invariant is the registry's: no capability without a check or a documented gap.

Everything mechanical is derived, so the table cannot rot:

npm run capabilities:sync     # rewrite the table from the live registry
npm run capabilities:check    # fail if it is stale (the Quality Gate's typecheck leg)
npm run dev:nodetool -- harness capabilities [--json] [--strict]

name, module, contract, suites and evals come from scripts/sync-capability-coverage.mjs, which reads the live specs, scans the agent suites, and imports the eval case files. The one hand-written field is gap, and the sync preserves it. A capability that names every other capability by construction — the category snapshot in capabilities-registry.test.ts, the belt walk in capabilities-coverage.test.ts — is excluded from suite attribution; counting them would make everything look covered.

Adding a capability means adding its check. Write the eval case or the suite first; npm run capabilities:check fails on a new capability with neither, and it will not accept a gap note that still says TODO. Changing what a capability declares — its description, input schema, category, or needsToolCallId — moves its contract fingerprint, and nodetool harness gate --base <ref> then refuses a diff that left the coverage mapping untouched: say which case covers the new contract, or write down why there isn't one. A refactor that leaves the contract alone demands nothing and runs the mapped checks as usual.

The core API is in-process (src/tools/mcp-tools.ts)

The workflow/node/job/asset tools call NodeTool's own code, never HTTP. There is no NODETOOL_API_URL, no fetch, and no server that has to be listening:

Concern	Where it comes from
Workflows, jobs, assets	@nodetool-ai/models (Workflow, Job, Asset)
Running / debugging a workflow	runWorkflow in @nodetool-ai/execution/service
Interactive escalations	submitEscalationVerdict + the debugSessions registry, same module
Debugging an app	runApplicationDebug, same module
Building an app	runApplicationBuild (src/app-build/build-service.ts)

@nodetool-ai/execution/service is the layer the REST routes call too, so a tool result and the endpoint's response are one function's answer and cannot drift. packages/websocket keeps the Fastify routes, auth and WS transport as thin adapters over it.

Three things live above this package in the dependency order and arrive by injection through getAllMcpTools(options):

• registry — a NodeRegistry. Node discovery needs it, and so does anything that executes. Without one those tools answer with a "no node registry in this process" error instead of reaching for a network fallback.
• examples — the shipped example-workflow catalog (JSON inside the installed node packages; only the server walks the metadata roots).
• exportDsl — workflowToDsl from @nodetool-ai/dsl.

The server builds all three in packages/websocket/src/mcp-tool-deps.ts and spreads mcpToolHostDeps() into every getAllMcpTools call site.

A fourth, providers, adds find_model and list_models — and only those. They enumerate the injected map, so without one they can say nothing but "no providers configured". The media tools (generate_image, generate_speech, …) used to be added beside them and are built-ins now: each reaches a provider through context.runProviderPrediction and reads nothing off the run, so the map was never their dependency. Gating them on it meant a host that injects none — a Code node, a JS script — got a belt that could critique_image and score_image_adherence but had no way to make an image, and nodetool.media.generateImage threw tool "generate_image" is not in this toolbelt after the run had paid for the prompt that produced its argument. Pinned by tests/sandbox-belt-reach.test.ts.

Script Voicing Tools (src/tools/script-voice-tools.ts)

The headless path from a written script to voiced takes and an assembled voiceover sequence. The editor voices a line over the chat WebSocket's generate_media / transcribe_audio RPCs, and the nodetool.script.* nodes do it inside a workflow; an agent outside the browser had neither. These call the provider directly, save each take as an asset, and write it back onto the persisted script.

Tool	Does
list_scripts	Scripts newest first, with line and voiced counts
get_script	Cast, lines, and each line's voicing status
voice_script_lines	TTS per line → a take, current on its line
assemble_script_timeline	Voiced takes → a saved timeline_sequences row

get_script reports the status the editor's gutter shows — draft (never voiced), stale (text or voice changed since the take), voiced, no_voice — and voice_script_lines defaults to every line that is draft or stale, so a whole script is one call. Each line uses its own voice (its override, else its speaker's) unless the call passes provider+model+voice to override them all; a half-specified override is an error, not a guess. Lines are voiced concurrently (default 3, max 8, 60 per call) and each take lands through a CAS on the row's updated_at.

Synthesis delegates to GenerateSpeechTool, so the encoded/streaming-PCM provider split is handled in one place. Word timings come from a best-effort ASR pass (whisper-1 by default, transcribe: false to skip it) and ride into the assembled clips as captions. Take duration is ffprobe's answer, falling back to the last word timing and then to the 3s placeholder — a take stays assemblable without an exact length.

The voice rule (effectiveVoice), the staleness rule (needsVoicing) and the script → timeline mapping (buildScriptTimeline) live in @nodetool-ai/timeline; the editor's "Send to timeline" and nodetool.script.ScriptToTimeline call the same functions, so the three surfaces cannot drift. Re-assembly rewrites this script's voiceover track in place and keeps clips other surfaces added.

Tests: tests/script-voice-tools.test.ts (in-memory DB, fake provider — no network).

Storyboard Render Tools (src/tools/storyboard-render-tools.ts)

The headless path from a directed storyboard to rendered media and an assembled cut. The editor has always had this path — the Storyboard surface builds a throwaway TextToImage → Output / ImageToVideo → Output graph per shot and runs it in the browser — but an agent outside the browser had to author, save, and run a workflow per shot, or drive the ui_storyboard_* tools, which only work while that board is open. These call the provider directly, save each result as an asset, and write it back onto the persisted board.

Tool	Does
list_storyboards	Boards newest first, with per-board still/clip counts
create_storyboard	Blank board (then edit_storyboard adds shots)
get_storyboard	Shots with ids, status, and whether each has a still/clip
direct_storyboard	Runs the Director over the board's brief/genre/style and writes the screenplay; redirect keeps retained shots' ids and media
render_storyboard_stills	text_to_image per shot → the shot's keyframe
render_storyboard_clips	the shot's clip: image_to_video seeded by the keyframe, or text_to_video from the prompt
revise_storyboard_clip	video_to_video revision of one shot's clip
assemble_storyboard_timeline	Rendered clips → a saved timeline_sequences row

edit_storyboard is the document half: beside the shot ops it carries the guided flow's set_setup, the scene ops (move_shot, duplicate_shot, update_scene, create_scene, merge_scene), set_style, and the version ops (select_version, delete_version, add_keyframe_version). Every one mirrors a ui_storyboard_* browser tool and writes the same document shape, so a board edited headlessly and one edited in the editor cannot diverge (PRD § 7.10).

Both render tools take targets (shot ids, indexes, or slugs) and default to "whatever still needs this step", so a whole board is one call. stale_only narrows that to the shots whose selected version was rendered from inputs the shot no longer has (isVersionStale), which is how a style change is re-rendered without re-rendering the board. Shots render concurrently (default 3, max 8, 24 shots per call). Every write is a CAS on the row's updated_at with a bounded retry, because concurrent renders all land on the same board document; a conflicting write re-reads and re-applies rather than clobbering.

The provider and model come from the call, else from the board's own imageModel / videoModel. There is no fallback default — an unset model is an error naming find_model, not silent spend on a model nobody chose.

Shot.render_mode decides whether a clip needs a still. "keyframe" (the default) animates the selected still; "direct" renders from the prompt and skips the stills step entirely, for shots where first-frame conditioning is the wrong trade — heavy motion, which I2V renders stiffer than the same model's T2V, and native-audio models, which are weakest on their image path. A direct shot's prompt carries framing and board style, since no still carries them. render_storyboard_clips takes mode to override every selected shot for one call without editing the board.

Prompt composition, entity seasoning (entitiesForShot, @nodetool-ai/protocol) and the shot → timeline mapping (buildStoryboardTimeline, @nodetool-ai/timeline) are the editor's, so a board rendered headlessly matches one rendered in the UI. Board entities are library assets carrying a metadata.nodetool_entity marker; their descriptors and first reference image ride along as the entities param, which the runtime expands at the provider layer.

Tests: tests/storyboard-render-tools.test.ts (in-memory DB, stubbed predictions — no provider calls).

Media Analysis (src/capabilities/analysis.ts, src/analysis/)

Five read-only capabilities that measure media instead of describing it: analyze_audio, analyze_audio_spectrum, detect_audio_events, analyze_video, detect_video_scenes. understand_video hands a clip to a multimodal model and gets prose; these hand back numbers a model cannot derive from watching — loudness against a delivery target, which octave band the energy is in, where the silences and onsets fall, how much the picture moves, where the cuts are.

The split is deliberate. src/analysis/audio-dsp.ts and video-frames.ts are pure functions over Float32Array samples and RGBA pixels — no decoder, no context, no I/O — so tests/audio-dsp.test.ts and tests/video-frames.test.ts check them against signals whose answers are known without running the code: a 1 kHz sine's spectral centroid is 1 kHz, a solid grey frame has zero contrast, and the BS.1770 reference signal reads -20.0 LUFS. media-decode.ts is the Mediabunny seam (PCM out of an audio track, RGBA frames at chosen timestamps, container metadata), and the capability file resolves the reference, picks the analysis parameters, and shapes the answer.

src/analysis/mediabunny-runtime.ts is where the Node codec adapter is registered, once per process. sandbox-av-media.ts imports it rather than keeping its own copy — two bootstraps racing to register the same adapter is the kind of thing that works until it does not.

Three properties worth keeping when changing any of this:

• No ffmpeg. Mediabunny decodes, so an install with no managed runtime tools still answers. nodetool.video.GetVideoInfo returns a record of zeros when ffprobe is missing; that is the failure mode this exists to avoid.
• Every answer is bounded, and says so. Series are decimated to a point budget (decimated), decoding stops at a duration cap (truncated), and video frames are never all held at once — forEachVideoFrame closes each one after the callback, because a few hundred 1080p RGBA frames is gigabytes.
• A measurement that means nothing says so. tempo.reliable is false below four onsets or below a confidence floor, and integrated_lufs is null rather than -Infinity for audio shorter than one 400 ms gated block.

That last one was earned. Spectral flux was raw rectified difference, and a held 1 kHz sine reported 48 onsets at a confident 87 BPM: at a hop that is not a whole number of cycles, a stationary tone's leakage pattern shifts between frames, so its bin magnitudes wobble forever. spectralFlux normalizes by the frame's own magnitude — the wobble is ~2%, a real onset is a large fraction of 1 — and detectOnsets carries an absolute minStrength floor beside its adaptive one, because where the curve is flat the local deviation collapses and every ripple clears mean + 1.5σ.

Timeline Frame Preview (src/timeline-preview/)

preview_timeline_frame composites a timeline at chosen timecodes and returns one image handle per frame. It is the only way an agent sees what a cut looks like: validate_timeline reads the document, ui_timeline_get_clip_frames samples one clip's source media in a browser, and neither shows the composited picture — the title over the shot, an animation mid-flight, a transition part way through, the track order deciding what covers what.

It resolves the scene through @nodetool-ai/timeline/scene (computeActiveLayers → resolveAnimatedLayerProps), rasterizes the text/shape/caption layers with the shared draw.ts rules, decodes media with Mediabunny, and composites with the shared drawTimelineFrame — so a previewed frame follows the same rules the editor's preview and the video export do. rasterize.ts is NodeRasterizer's twin, differing only in returning the @napi-rs/canvas surface instead of an RGBA buffer, because Canvas 2D draws it rather than uploading it.

Three properties worth keeping:

• No GPU, no ffmpeg, no browser. It imports @nodetool-ai/timeline/scene, not /render, so TypeGPU never enters this package's graph. That is what lets the capability run wherever the agent runs — the headless kernel, the CLI, a sandboxed CI box with no Vulkan ICD.
• Frames are handles, not pixels. view_image is the one mechanism that pulls pixels into context, so each frame is persisted with persistOutput and returned as an asset_id. Three inline base64 frames would sit in every later turn of the conversation whether or not the model looked at them.
• What it cannot draw, it names. Color and blur adjustments map onto the canvas filter; chroma key, vignette and sharpen have no Canvas 2D form and come back in effects_not_applied. A layer that drew nothing carries a skipped reason — an unrendered draft clip, an unreadable asset, no decodable frame at that source time — rather than vanishing from the report.

Tests: tests/capabilities-timeline-preview.test.ts, which reads pixels off the rendered PNGs (layer order, a fade's mid-ramp opacity, text actually drawn) rather than asserting a call happened.

Google Workspace Tools (src/tools/google-workspace-tools.ts)

Drive, Gmail, Docs, Sheets and Calendar tools that authenticate with the access token from the user's Google sign-in — there is no API key. The Supabase Google login hands the browser a provider_token, the web app posts it to POST /api/oauth/google/session, and the server stores it as an OAuthCredential under provider google. Tools read it back through the virtual secret key GOOGLE_ACCESS_TOKEN, which getSecret routes to resolveGoogleAccessToken (refreshing a stale token when GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET are set).

They are not in BUILTIN_TOOL_CLASSES. A server without a login can never produce a token, so the chat toolbelt adds them only when isGoogleWorkspaceEnabled() (@nodetool-ai/config) is true — Supabase auth mode, or NODETOOL_GOOGLE_WORKSPACE=1.

The fourteen lib.google.* nodes are gone. Each wrapped a call this module already makes, on the same sign-in token, and the module makes six more no node ever offered: get one Drive file, get one Gmail message, list labels, create a spreadsheet, list calendars, delete an event. The capability is the only Google surface now, and it is two things at once — an agent tool and a guest import.

// Nothing to install — capability modules are mounted by the host, not
// resolved as packs from the catalog.
import { drive_search, gmail_send_message } from "@nodetool-ai/sandbox-nodetool/google";

const { files } = await drive_search({ query: "name contains 'invoice'" });
await gmail_send_message({ to: "a@b.c", subject: "Invoices", body: String(files.length) });

import { getGoogleWorkspaceTools } from "@nodetool-ai/agents";

if (isGoogleWorkspaceEnabled()) {
  toolbelt.push(...getGoogleWorkspaceTools());
}

A missing or revoked credential surfaces as { error } telling the user to sign in with Google again, rather than throwing — the agent can then pick another route instead of failing the whole step.

Where the permission gate is set

One ladder decides every actionable call. decidePermission (tools/tool-permissions.ts) reads a mode and a category, gatedCall (capabilities/invoke.ts) runs the sequence around it (read-class fast path, mode decision, session allow-set, approval round trip), and gateTools (capabilities/gate-tools.ts) is the door a Tool walks through it. What differs between hosts is who answers when the ladder asks.

A host publishes one PermissionGateOptions under PERMISSION_GATE_CONTEXT_KEY (src/types.ts), and every loop under it reads that object with gateFromContext instead of building a gate of its own (invariant I-1). This is the channel RunBudget already travels on, for the same reason: ProcessingContext.copy() shallow-copies the variable bag, so a child shares the host's gate object rather than a clone, which is what makes a mid-turn set_permission_mode and an "allow for this chat" answer reach a loop that started before them. The contract itself — the key, the types, decidePermission, headlessGate — lives in @nodetool-ai/runtime (permission-gate.ts) so the workflow hosts below this package can set a gate; this package re-exports the names. Every host sets one, so a context carrying no gate is a bug: gateFromContext logs it at error level and answers a gate that lets read-class calls through and denies everything else (mode default, approver deny). A host that means "nobody to ask" says so with headlessGate(hostName).

Host	Where its gate comes from	Mode	Approver
Chat turn (web, Telegram)	chat-turn.ts sets chatGate on the turn context	the thread's live mode, read through a getter so set_permission_mode lands mid-turn	round trip to the client (requestToolApproval)
MCP mount	mcpSessionGate() in mcp-agent-tools.ts — headlessGate("MCP") with execute_code seeded into sessionAllow — and the whole belt wrapped in gateTools, so the direct registrations and tools.<name>() inside a code action meet the same ladder	auto: the client's user connected this agent deliberately, and an MCP session has no approval UI of its own to prompt through	nobody: the headless deny. The one standing approval is execute_code, because the client already put that call, code included, in front of its user; an escalation from inside a run (an un-allowlisted Apify actor) has no such answer and is denied
run_node child	runSingleNode builds a context of its own, so the turn's gate is passed in as an argument and set on it	the calling turn's, the same object	the calling turn's client
AgentNode in a workflow	genProcess wraps what buildTools returned in gateFromContext(context, "Agent node")	the host's, or auto when no host set one	the host's, or the headless deny
JS script	js-script-sandbox.ts passes gateFromContext(context, "JS script") to createCapabilityRun	same	same
nodetool.code.Code node	code-node.ts reads gateFromContext(context, "Code node") once and uses it for both doors: createCapabilityRun for the @nodetool-ai/sandbox-nodetool/* imports, gateTools around the belt the bridge calls	same	same
Kernel workflow run	buildWorkspaceExecutionContext (packages/execution/src/service/workflow-workspace.ts) sets headlessGate("kernel workflow run") on the context it builds, and ExecutionSession.create sets the same on a caller's own context when that caller set none (see below)	auto	nobody: the headless deny
Headless job runner (trigger-driven runs)	packages/websocket/src/headless-job-runner.ts sets headlessGate("headless job runner") on the run context	auto	nobody: the headless deny
nodetool agent run on a TTY	createCliPermissionGate (packages/cli/src/permission-gate.ts), host name nodetool agent run, set on the run's context	--permission-mode, defaulting to default	the terminal: y / n / a prompted on stderr, because stdout carries the result. a answers allow_for_chat, and the name lands in the run's shared sessionAllow
nodetool agent run with the objective piped in	the same builder, not interactive	--permission-mode, defaulting to auto	nobody: headlessGate's deny, its reason printed once per run
nodetool-chat reading piped stdin (runStdinMode)	the same builder, host name nodetool-chat, set on each line's context	--permission-mode, defaulting to auto	nobody: the headless deny
nodetool-chat --url	no gate is built here	—	the server's chat turn, which gates the belt it runs
nodetool-chat interactive session	nothing: the Ink session builds its own belt in app.tsx, wraps it in no gateTools, and puts no key on its context	none	nobody

A CLI run that cannot prompt keeps the mode it was asked for and changes only the approver, so a piped --permission-mode plan still blocks what plan mode blocks. The sessionAllow Set is created once per run and shared by reference, which is what makes a outlive the call that answered it: nodetool-chat rebuilds its belt for every input line and consults the same Set. Under --json the notice and any prompt ride the event stream rather than stderr, so that stream stays one JSON object per line.

The interactive CLI session is the host A2 left open. Its belt is RunSubtaskTool plus raw tools, so nothing there meets the ladder. The reason is the terminal, not the design: the Ink session reads stdin itself (useInput), so the readline prompt every other CLI path uses cannot share it, and an approval there needs an Ink component instead. Rather than accept a mode it would ignore, packages/cli/src/index.ts prints that --permission-mode does not apply to the interactive session.

Both gated CLI hosts wrap their belt the way chat does. gateTools covers the belt, and execute_plan is wrapped on its own, because toolForCapabilityName builds a LazyCapabilityTool whose process calls the implementation directly: a gate on the delegation run covers what that implementation invokes, not the tool itself. The delegation primitives (run_subtask, start_subtask, wait_subtasks, run_search, create_plan) stay ungated in both hosts, because spawning a child loop has no effect of its own and the child acts through the belt that is already gated.

The kernel row is a decision, set explicitly. A workflow run is consent: the user pressed Run on a graph whose nodes list their tools, so an agent loop inside it runs auto with nobody to ask. buildWorkspaceExecutionContext says so by setting headlessGate from @nodetool-ai/runtime, and ExecutionSession.create does the same for a caller that hands in its own context with no gate on it (nodetool run, the WebSocket job runner, the app simulator), while a caller whose context already carries a gate — a chat turn's run_node — keeps it by reference. Setting it at the choke point is what lets gateFromContext treat an absent gate as a bug and fail closed.

Plan mode already blocks run_node one level above the node. run_node is classified execute, and decidePermission("plan", "execute") is block, so a chat in plan mode gets blocked_in_plan_mode from the belt before the node runs. The gate on the node closes the same hole reached another way: a set_permission_mode to plan while a node is already running, or any other host that puts a plan-mode gate on a context. What the child inherits is pinned by packages/websocket/tests/chat-turn-handler-run-node.test.ts, which asks gateFromContext from inside the node's own executor.

Control tools and submit_result stay ungated on an AgentNode. They are graph wiring rather than capabilities: a control tool fires an outgoing control edge, and submit_result hands the node's structured outputs back to the node. Both are appended after gateAgentTools has run. The wrap sits at the genProcess call site rather than inside buildTools because buildTools is the hook a subclass overrides, and gating inside it would be skipped by any override that does not call super.

The headless gate denies, and names the host that had nobody to ask. headlessGate(hostName) runs auto with an approver that resolves "deny" and logs the refusal. In auto the ladder allows read, write, execute and external outright, so that approver is reached only by an escalation the ladder itself raises: admitCodeAction asks before a high-risk execute_code action in auto, and the Apify actor policy asks before an actor the install has not allowlisted. Resolving "allow" would grant what those escalations exist to withhold, and never resolving would hang the run (invariant I-4). headlessDenialReason is exported so a host that runs headless on purpose can print the same sentence once at the start instead of once per denied call.

Three construction sites may still build an ungated run. capabilities/lazy-tool.ts and tools/serp-tool-factory.ts build a Tool the host gates from outside with gateTools; capabilities/packs.ts reads a SKILL.md, a read-class call with nothing for the ladder to withhold. packages/agents/tests/gate-from-context.test.ts walks every packages/*/src for ungatedCapabilityRun and fails on any file its allowlist does not cover. It also asserts that the three sites it names by hand were found, so it cannot pass on a walk that matched nothing. The walk used to stop at this package, which is why the Code node's ungated mount sat outside it.

Approving a plan is the permission gate

There is no plan-approval callback. execute_plan is classified external in tools/tool-permissions.ts, so running a plan is approved where every other action is approved — one ladder, one dialog, one mode (set_permission_mode) — instead of a second round trip the host had to wire by hand. create_plan produces a plan and stops, so the user sees what will run before anything does.

A host with nobody to ask fails closed at execute_plan rather than inside the loop, which is what invariant I-4 asks for.

Parallel Task Execution

create_plan decomposes an objective into parallel tasks via TaskPlanner.planMultiTask(), and execute_plan runs them. Tasks form a DAG — independent tasks run concurrently.

How It Works

1. Planning: LLM generates a TaskPlan with multiple Task objects, each with dependsOn arrays
2. Scheduling: ParallelTaskExecutor hands the tasks to scheduleDag (utils/dag-scheduler.ts), which starts a task the moment its last dependency settles. TaskExecutor schedules its steps the same way.
3. Merging: createDynamicMerge() interleaves the streams and admits a newly-started node into the running merge (utils/merge-generators.ts)
4. Completion: A task's result reaches shared memory and its terminal task_update is emitted before its dependents are released; the stream ends when every task has settled

Scheduling is event-driven, not round-based. The executors used to dispatch in barrier rounds — start every ready node, wait for the slowest, recompute — so a plan ran as slow as its slowest sibling in every round, and the cap on rounds per plan and per task turned depth into failure. Both caps are gone: what bounds the work is the run budget, maxStepIterations per step, and the permit pool. A cycle never reaches the scheduler (PlanBuilder rejects it), and a node nothing could ever run settles failed with unsatisfiable dependency rather than hanging.

The aggregation step is the finish step's dependency on every other step in the task, not a deferral rule: it runs last because it waits for its siblings, and a sibling that fails blocks it (I-5) instead of aggregating over a hole.

Task Plan Structure

interface TaskPlan {
  title: string;
  tasks: Task[];           // Multiple tasks forming a DAG
}

interface Task {
  id: string;
  title: string;
  steps: Step[];
  dependsOn?: string[];    // Task IDs this depends on ([] = independent)
}

interface Step {
  id: string;
  instructions: string;
  dependsOn: string[];     // Step IDs within this task
  tools?: string[];        // Restrict available tools
  outputSchema?: string;   // JSON schema for step result
}

Skipping Planning

execute_plan takes the tasks as its argument, so a caller that already knows the decomposition passes it straight in and never calls create_plan. The shape is the TaskPlan above with snake_case keys (depends_on), validated by buildPlanFromTasks (tools/plan-builder-tools.ts) before anything runs.

Executor defaults (constants.ts)

Two bounds are left, both in constants.ts, both read by parallel-task-executor.ts and task-executor.ts when a caller names none: DEFAULT_MAX_STEP_ITERATIONS (action rounds per step) and DEFAULT_MAX_CONCURRENT_AGENTS. The round caps that sat beside them are gone (see the scheduling note above); the rest of what bounds a run is RunBudget below.

Nothing resolves a per-run policy object: execute_plan passes the run's budget and the parent's per-step iteration cap directly.

maxConcurrentAgents bounds one merge numerically (tasks per plan, steps per task), not the run. The run-level bound is the budget's permit pool below, and a permit stands for one open provider conversation, not one layer: a task takes none, each step holds one through branchPool (subagent.ts), and a child a step or sub-agent spawns runs on its parent's permit while the parent is idle in the spawning call and borrows from the run pool beyond that. A holder never queues for a second permit, so nesting cannot deadlock, and one pool bounds tasks, steps and sub-agents together (run-budget-propagation.test.ts, "bounds every layer"). start_subtask refuses past MAX_BACKGROUND_SUBTASKS_PER_TURN (tools/start-subtask-tool.ts) with background_limit_reached; a detached child still on its parent's permit can overlap the parent's next turn. Planning draws on the same budget: TaskPlanner takes a budget (falling back to the context's) and a refused turn stops planning naming the budget's reason.

One budget per run (@nodetool-ai/runtime → RunBudget)

A run's bounds are one object, created by the host and shared downward, never re-created by a child. RunBudget carries four:

Bound	What refuses	Setting
turns	a turn whose worst case would cross the USD cap	NODETOOL_AGENT_TURN_COST_CAP_USD
deadline	any turn or tool call after the wall clock runs out	NODETOOL_AGENT_TURN_DEADLINE_MS
concurrency	a permit past the run's pool — tasks, steps, nested executors and sub-agents all draw from it, a holder's own permit counting as one	NODETOOL_AGENT_MAX_CONCURRENCY
turnCount	a turn past the run's cumulative total	NODETOOL_AGENT_MAX_TURNS

Spend admission is reserve-then-commit, because the next call's cost is only known afterwards: $0.49 spent under a $0.50 cap still admits a $0.30 call. turns.reserve is the single admission point and records the first reason it refuses; exhausted is set once, so a run that ran out of money and then ran out of time stopped for the money.

A reservation is a handle, because the budget is shared. turns.reserve answers a TurnReservationHandle carrying that turn's own worst case and its own pricing (or null when it refused), and turns.commit(handle, actualUsd) settles exactly that one — every other loop's reservation stays outstanding. One shared counter released whatever was in flight, so a chat turn committing its turn handed a concurrent sub-agent's headroom back and the cap stopped holding; a single unpriced flag likewise booked whichever turn committed first under the other's pricing.

An unpriced model is not a free one. A model the price catalog does not cover has no worst case to reserve, so it is admitted only while its prompt fits NODETOOL_AGENT_UNPRICED_TOKEN_CEILING, counted in unpricedTurns, and leaves spentUsd a declared lower bound — the unpriced convention nodetool costs already follows. With no USD cap configured there is nothing for the ceiling to protect and it does not apply; the run stays bounded by its deadline and turn count.

How a child reserves against its parent. The budget rides CapabilityRun.budget and SubAgentToolRuntime.budget, and subAgentRuntime() (capabilities/agents.ts) is the one seam where it joins the runtime, so every delegation capability inherits it from a single place. runSubAgent hands that object — never a new one — to its CodeActExecutor; execute_plan carries it into ParallelTaskExecutor, which forwards it to every TaskExecutor and on to every CodeAct step. It also rides RUN_BUDGET_CONTEXT_KEY on the context, which is how a loop the host never constructs finds it: an AgentNode started through run_node, or a JS script. ProcessingContext.copy() shallow-copies the variable bag, so a child context shares the same budget object rather than a clone — that sharing is what makes a cap a run total.

Permits are taken once per branch. A run-total semaphore deadlocks if a holder queues for a second permit, which nested merges do by construction: ParallelTaskExecutor holds permits for tasks whose TaskExecutors then queue for step permits. acquireRunSlot is therefore re-entrant on a context flag, and each DAG executor holds one slot for the length of its run.

A budget stop is visible, never silent. generateLoop used to return, so a consumer could not tell a refused turn from the model ending its turn. It yields one ProviderStop as its final item — budget, deadline, iterations, or aborted — and a turn the model ends on its own yields none. Every consumer surfaces it: a CodeAct step fails naming the reason the budget recorded, and AgentNode raises a node error. classifyProviderStream (llm-nodes/agent-utils.ts) yields a stop event for it; before that branch existed the item fell through its chunk/tool-call/message chain and vanished.

Absent means unbudgeted, not exhausted: a kernel workflow run with no host budget behaves exactly as it did before.

Step failure is terminal, not completion

A step that fails sets step.failed + step.error and leaves completed false, and its step_result carries the protocol-level error field. Nothing downstream may treat a failure as a satisfied dependency: the scheduler walks the failed node's transitive dependents and settles each as failed, naming the dependency directly above it, and execute_plan answers a plan whose every task failed with error: "plan_failed" and the per-task failures rather than compiling a deliverable out of nothing; it does not throw. The same holds one level up — a failed task blocks its dependent tasks — and a step or task the run's signal cut short settles once as "Step aborted" / "Task aborted", never as "dependency failed", rather than being left looking like it is still running.

CodeAct Execution (src/codeact/)

The action space of the step loop, and the only one. Each step acts by writing JavaScript that runs in the QuickJS sandbox with the toolbelt exposed as imports from @nodetool-ai/sandbox-nodetool/<namespace>, finish(result) for host-validated completion, and memory (nodetool.memory.*) for results a later action or turn needs — there is no cross-action variable bag. The prompt tells the model to record each generate/speak result (an asset:// uri) with nodetool.memory.save and reuse it; local variables die with the action, and return is the observation only. A step action runs with no guest fetch and no secret scope: network and secrets are reached only through the gated capabilities, the same rule the chat session's actions run under. Design and the research it follows (CodeAct, ICML 2024): docs/codeact-design.md.

• CodeActExecutor keeps the message contract, memory writes, and failure semantics the step loop has always had — consumers work unchanged. Bridged tool calls surface as tool_call_update events (ids codeact_<n>).
• The core set goes top level. Every NodeTool tool that mirrors a Claude Agent SDK built-in (CORE_TOOL_NAMES in @nodetool-ai/runtime — the file set, glob/grep, web_search, browser/http_request/download_file, todo_write, run_subtask) is offered to the provider as an ordinary tool next to execute_code, for every provider. Those are the shapes models are trained on, so a tool call beats a sandbox round trip that only forwards one. They stay on the belt — nodetool.web, nodetool.agents and hand-written fan-out call them from code — but the prompt documents them once, under "Direct tools", instead of as a catalog signature. splitCoreTools / buildCoreProviderTools in src/codeact/tool-api.ts.
• Discovery goes top level with it. DISCOVERY_TOOL_NAMES (find_model, list_models, list_provider_models, search_nodes, get_node_info, list_nodes) answers which providers, models and node types this install has — one question with one answer, and a wrong answer is a hallucinated model id that fails at generation time, after the run was paid for. The two sets stay apart because only the core set has SDK built-ins behind it: DIRECT_TOOL_NAMES is their union and is what the four offer sites read (splitCoreTools, the CLI turn, the websocket runner, and the MCP mount), while SDK_NATIVE_TOOL_REPLACEMENTS still reads CORE_TOOL_NAMES alone. nodetool.models.pick and nodetool.nodes.search are unchanged — the belt keeps every tool, so an action still composes them.
• The MCP mount subtracts. It offers DIRECT_TOOL_NAMES minus every key of SDK_NATIVE_TOOL_REPLACEMENTS, because an MCP client (Claude Code, ChatGPT) is the host agent that table describes: offering NodeTool's workspace-scoped read_file beside the client's own would put two tools of one name and two different roots in front of one model. What survives has no host equivalent — discovery, the server-side reach that runs behind NodeTool's SSRF guard and secrets (browser, http_request, download_file, list_directory), and run_subtask, whose child gets the NodeTool belt rather than the client's. It is a belt.filter, so a session missing an injected dependency (no node registry, say) advertises no node tools instead of a tool that cannot run. Before this the mount registered only execute_code and view_image: everything was still reachable as an import inside an action, but every discovery question cost a sandbox round trip, which is the tax the direct set exists to remove.
• On claude_agent_sdk the built-in wins outright. The provider drops every tool SDK_NATIVE_TOOL_REPLACEMENTS maps (read_file→Read, write_file→Write, edit_file→Edit, glob→Glob, grep→Grep, web_search→WebSearch, todo_write→TodoWrite) from its MCP toolset and lets the SDK's own tool serve the call. The path-scoped five are substituted only when the caller passes a workspaceDir, which becomes the session cwd — without it the SDK would resolve paths outside the run's workspace, so NodeTool's contained versions stay. list_directory, browser, http_request, download_file and run_subtask are never substituted: no built-in covers what they do (Task would hand the child SDK tools, not the NodeTool belt). See packages/runtime/src/providers/core-tools.ts.
• Progressive disclosure: resident tools (CODEACT_RESIDENT_TOOL_NAMES — the search family incl. web_search/search_nodes/run_search/ asset_search/grep/glob, the Claude-agent file set (read_file/write_file/edit_file/list_directory), browser, HTTP, memory, run_subtask) are documented in full; past CODEACT_DEFER_THRESHOLD tools, the rest is name-only in the prompt and discovered in-sandbox with await nodetool.searchTools("query") (ToolSearch grammar). All tools stay callable either way. Discovery lives on the object model and nowhere else: the bare searchTools() global is gone, because a model that knows nodetool.* looked for it there and burned two rounds finding out it was elsewhere.
• Imports are discoverable too: nodetool.packs.list() reports every installed pack and whether this session allows it, modules(pack) the specifiers it declares, exports(specifier) the function names one module exports, and docs(specifier) the pack's SKILL.md. A pack installed but off the allowlist is listed as allowed: false rather than hidden — hiding it teaches the model the pack does not exist. The data reaches the guest as a tool (list_sandbox_packages, capabilities/packs.ts), which is the only path the object model has: it owns no host bridge. Exports are exact for host modules (SANDBOX_HOST_MODULES), WASM modules (the manifest contract) and platform modules (the capability registry); for a guest JS module they are read off its own export statements with acorn, and a module that re-exports with export * answers complete: false instead of a short list.
• Every step executor is one: TaskExecutor, ParallelTaskExecutor, run_subtask, and run_search all construct CodeActExecutor. StepExecutor — the older one-JSON-tool-call loop — is no longer exported; two callers keep it because they are one-shot structured verdicts on a fail-closed path where a sandbox error would only add a failure mode: SupervisorAgent and the app-build spec stage.
• Chat turns run in it too: the websocket runner swaps the toolbelt for execute_code (+ the core set, + view_image) via createChatCodeActSession (src/codeact/chat-codeact.ts), which routes every imported belt name to the chat runner's own tool router instead of buildToolBridge — permission gating and client (ui_*) round-trips stay where they are. When the belt carries the ui_* workflow document tools, actions also get the graph object model (src/codeact/graph-model.ts): openWorkflow() returns a model whose synchronous mutators queue ops against a local mirror and commit() replays them through the same ui_* contract. The CLI's local (no-server) turn runs the same session — execute_code, the core tools and view_image are what processChat sees, wired in packages/cli/src/chat-codeact.ts.
• Both executors also load the nodetool object model (src/codeact/nodetool-api.ts): the platform as objects instead of raw imports — nodetool.workflows (list/get/run/start/debug/validate/ create/open), nodetool.batch(items, fn, {concurrency}) for bounded fan-out (run a workflow once per CSV row), nodetool.models (pick(capability) resolves one ranked model; find/list for the long form; forProvider(provider) for one provider's own catalog), and nodetool.media (generateImage/editImage/generateVideo/animateImage/speak/transcribe/embed plus the judge loop critique/compare/scoreAdherence and understandVideo(video, prompt, model), which hands a whole clip to a model that reads video and answers as text — each taking a pick/find result or "provider/model_id"; plus the host binaries ffprobe(path), ffmpeg(args) and downloadVideo(url, outputFile), whose argv is confined to the workspace and to local-file inputs by src/host-binary-guard.ts and bounded on wall clock, captured output, artifact size and concurrency by src/host-binaries.ts), nodetool.nodes (search/info/list — the graph builder's discovery half), nodetool.documents (convert, PDF text/tables, markdown↔pdf), nodetool.apps (build/debug), nodetool.agents (run(prompt) spawns a run_subtask child with a fresh context; fan out via nodetool.batch(prompts, (p) => nodetool.agents.run(p))), the single-node harness on nodetool.nodes.run(type, inputs), nodetool.web (the outside world: search(query, {provider}), news and images are all one routed web_search with a search_type, which picks the first configured backend host-side; provider pins one: "default", "serpapi", "dataforseo", "brave", "apify", "openai", "google" — plus browse(url), fetch(url), download, screenshot), nodetool.memory (save/list/search/update/remove over memory_*), nodetool.shared (list/read/publish over list_shared/read_shared/share_result — the run's own scratchpad, beside thread-scoped nodetool.memory), nodetool.threads (list/get/last/message over list_threads/get_thread/ get_message — the chat history itself, read-only), nodetool.style (profile/record), nodetool.email (search/archive/label), plus assets (list/search/images/get/save/read), jobs (with wait(id, {timeoutMs, pollMs}) polling a background job to settlement), collections (full RAG loop: index/indexBatch/search/hybridSearch/query), timelines, sketches, scripts, and storyboards. workflows also carries resolve(sessionId, escalationId, action) for interactive-run escalations and example("<package>/<name>") feeding copyFrom (list({workflow_type: "example"}) enumerates the shipped examples). Every method wraps a belt tool, so gating and routing are untouched; a method whose backing tool is missing throws naming the tool, and the prompt section documents only the namespaces the belt can serve (buildNodetoolApiPromptSection). One surface per capability: tools the object model wraps (nodetoolApiCoveredToolNames, plus GRAPH_MODEL_TOOL_NAMES when the graph model loads) are filtered out of the prompt's tool catalog — they stay callable through the bridge and findable via nodetool.searchTools(), but the nodetool.* form is the only documented one. Workspace files are the deliberate exception: they are not wrapped, because the sandbox's own workspace.* API is in-process and costs no tool call — the action contract steers there.
• The belt carries only what a model cannot write itself. The pure-computation tools (calculate, geometry, trigonometry, statistics, unit_conversion) were deleted outright, MCP included, and so were the code tools run_code and js — execute_code is the code surface, and a second one only invited the model to run code without the sandbox's nodetool.* API. The nine provider-specific duplicates went next. The media four — image_generation, openai_image_generation, google_image_generation, openai_text_to_speech — were deleted, because nodetool.media covers them through the provider-agnostic generate_image / generate_speech. The search five — openai_web_search, google_grounded_search, dataforseo_search, dataforseo_news, dataforseo_images — were never duplicates at all: they are the backends the single web_search capability routes to, so they became plain functions with no wire name (backend still pins one). google_news and google_images later folded into it too, as search_type: "news" | "images" — three wire names and three routing tables for one question with a parameter on it, which also left Brave and Apify reachable from no tool at all. Image search split back out into its own image_search capability (web.ts's imageSearchImpl is webSearchImpl with search_type pinned to "images" — one routing table, a second name): a model reaching for an image search reliably guesses a name like images before it guesses a parameter on a text-search call, and a guess against the "web" module's export list is exactly what synonymHintSentence in codeact/capability-modules.ts corrects. News stayed a search_type, since nothing observed a model reaching for a news_search import the same way. Every host therefore assembles the one belt getBuiltinTools() (src/tools/builtin-tools.ts) returns; a saved AgentNode naming a retired tool resolves to its replacement through RETIRED_TOOL_NAMES in @nodetool-ai/llm-nodes.
• Authoring a graph is a package, not a builder. nodetool.graph() — a builder taking every node type as a free-form string — is gone; a session authors with @nodetool-ai/sandbox-dsl, which ships one generated function per node type with that node's real inputs, one module per namespace. A type the catalog does not have has no export, so a hallucinated type fails at import instead of surviving into a graph nobody checked. withGraphDslPackage (src/codeact/graph-dsl-package.ts) is the wiring: it puts the pack on the session allowlist when the belt carries create_workflow, validate_workflow and run_workflow and the catalog serves the pack, so a machine without it installed is never told to import it. CodeActExecutor and createChatCodeActSession both call it, and the answer also gates GRAPH_DSL_PROMPT_SECTION. Consent is per pack: an allowlist entry covers that specifier's subpaths, which is what makes seventy-two namespace modules one prompt line. Tests: tests/codeact-graph-dsl.test.ts.
• Eval suite codeact scores the executor on offline instrumented cases: nodetool eval codeact -p <p> -m <m>. Beyond the four toy-toolbelt cases it covers the full nodetool.* API surface over two deterministic in-memory worlds (src/evals/codeact-api-core.ts, codeact-api-surfaces.ts) whose fakes are named like real belt tools so the object-model prelude lights up. tests/codeact-api-coverage.test.ts fails when a namespace loses its last case. Further cases (src/evals/codeact-sandbox-pack-cases.ts) cover sandbox packages: importing a pack the session allows and computing from what it parses, reading a pack's SKILL.md through get_sandbox_package_docs instead of guessing its API, reading the catalog instead of guessing what is installed, reporting a pack as unavailable rather than working around it, and using two packs in one action. A case names its allowlist in sandboxPackages, and the runner puts a catalog over the shipped packs (shippedPackCatalog()) on the context: the three host packs plus @nodetool-ai/sandbox-dsl and @nodetool-ai/sandbox-flow, whose modules are guest JavaScript the packs ship outright — a pack with an npm dependency would need @nodetool-ai/sandbox-compiler, which this package does not depend on. requiredSessionTools scores tools the executor adds rather than the case, which the recorder cannot see. scripts/dump-codeact-run.ts <case> <provider> <model> replays one case live and writes every action's code to nodetool-debug/ — the tool to reach for before touching the action-contract prompt. Run it with IS_SANDBOX=1 … --max-iterations 40. The residual misses rotate with sampling, so judge a prompt change on the per-action dumps, not on one run's pass count.
• Tests: tests/codeact-executor.test.ts, tests/codeact-eval.test.ts, tests/chat-codeact.test.ts, tests/nodetool-api.test.ts and tests/nodetool-api-*.test.ts (scripted provider, real sandbox, no network).

create_plan / execute_plan — the chat's plan mode

The chat's Plan permission mode blocks every mutating capability, so a multi-step request used to come back as prose. create_plan (in the agents capability module) runs TaskPlanner.planMultiTask over the objective, forwards the planning_update / task_update events into the thread — the web already renders both — and returns the plan's shape. It executes nothing: the planner produces a plan, and the turn ends with it on screen.

It is offered only in plan mode, and there it joins DIRECT_TOOL_NAMES so the model can call it outright; a deliverable reachable only by writing a sandbox action to import it is one the model does not reach for. The planner sees the parent belt minus create_plan itself, so steps route to real tool names and no plan contains "call create_plan".

execute_plan is the other half: it takes that plan back and runs it on ParallelTaskExecutor. The plan travels inline — title plus the tasks array the model copies out of create_plan's result. There is no plan store and no plan id: passing the plan itself is what survives the mode switch, keeps what runs identical to what the user is looking at, and lets "run it, but drop task 3" be expressible. Plans are a few hundred tokens.

The plan is checked before anything runs, by PlanBuilder — one authority for what a plan may be, so a plan create_plan produced cannot be rejected here. buildPlanFromTasks (tools/plan-builder-tools.ts) sorts the tasks so every dependency precedes its dependents (a hand-edited plan need not arrive that way) and feeds them through addTask/finish. Kahn's residue is exactly the tasks in a cycle, so a cycle is reported by name rather than as a task that was "not added yet". A rejection returns invalid_plan with one issue per offender and runs nothing.

Every message the executor yields is forwarded verbatim — the task_update events are what the thread and the sidebar render, so re-summarizing them would leave the user watching nothing until the end. The call returns each task's completed/failed (with the step and error behind a failure), a results map keyed by task id, and beside it a steps map of per-step results, so a caller can read what one step produced without unpacking its task. A plan whose every task failed comes back as error: "plan_failed" with those failures, not as a throw. The step results also stay in the run's shared memory under task:<id>, which the description points at so the model reads one back with read_shared instead of redoing the work.

execute_plan is classified external: running a plan is not one action but every action in it. That is what blocks it in plan mode — the permission gate is the confirmation, and there is no second approval step — and asks once everywhere else. Each step's own tool calls stay gated inside its child loop. The steps see the parent belt minus both plan capabilities: a step that re-plans or re-runs the plan it belongs to is a loop.

In chat-turn.ts it sits on the belt in every mode, gated, so a call in plan mode answers blocked_in_plan_mode rather than "unknown tool"; it joins DIRECT_TOOL_NAMES in every mode except plan.

Sub-Agent Core (src/subagent.ts)

The one place that knows how to spawn, stream, and settle a child agent. A sub-agent is an async generator of ProcessingMessage events whose return value is how the run settled — CodeAct is the default producer, but anything with that shape (a future reviewer, a researcher) streams through the same pipe and nests in the UI the same way.

Primitive	Does
runSubAgent(opts)	One CodeAct child loop: single-step task, optional outputSchema (structured via finish(), prose otherwise), yields events, returns SubAgentOutcome — never throws for run failures
settleStepResult(sr, {hasOutputSchema})	The unified failure detection: top-level step_result.error, the sole-key {error} payload a dying step reports, and (schemaless only) any string error property
forwardSubAgentStream(gen, opts)	Drives any sub-agent generator: tags events (parent_tool_call_id, subtask_depth), forwards without letting a broken forwarder kill the child, honors an abort signal between events
enterSubAgentDepth(ctx, maxDepth)	The shared recursion gate over SUBTASK_DEPTH_KEY: refuses past the cap, else returns a copied context with the depth bumped
SubAgentTool	Base class for tools that expose a sub-agent to a parent model — subclasses declare the tool surface, translate params into a SubAgentToolRun, and pick the child toolset; the base owns depth gate, streaming, tagging, settlement

Every spawn site goes through it: RunSubtaskTool (inherits the full parent belt, stitches itself in for recursion), RunSearchTool (read-only allowlist, breadth-scaled iteration budget), and StartSubtaskTool (same child but returns a subtask_id receipt immediately so the parent keeps its turn) are thin SubAgentTool subclasses. WaitSubtasksTool reads the same per-turn BackgroundSubtaskRegistry (src/background-subtasks.ts) that the background pump settles — wait_subtasks({ids?, timeout_ms?}) blocks until every requested record left "running" (or on timeout/abort, which return current statuses instead of throwing). The registry lives on SubAgentToolRuntime.background, one per chat turn, built by the host (websocket runner, CLI); a host with no registry refuses by name. A background subtask's events still stream to the UI tagged with parent_tool_call_id/subtask_depth, and nesting depth is still gated by enterSubAgentDepth. In CodeAct: await nodetool.agents.start(prompt)

• await nodetool.agents.wait({ids, timeoutMs}) — fan out while you keep working. A generator that is not a CodeAct child streams through forwardSubAgentStream directly. A new delegation tool should be another subclass, not another copy of the machinery.

Tests: tests/subagent.test.ts (pure-function coverage), plus the spawn-site suites (tests/run-subtask-tool.test.ts, tests/run-search-tool.test.ts, tests/background-subtasks.test.ts for start/wait and the registry, tests/capabilities-agents.test.ts through the capability seam) which exercise the core end-to-end.

Code-shaped orchestration is CodeAct, not a mode

There is no script mode. It was a third planning mode beside TaskPlan and the graph planner: ScriptPlanner had the LLM author one JavaScript orchestration script and ScriptRunner executed it in the sandbox, spawning a sub-agent per agent() call. CodeAct made it redundant — a step already acts by writing JavaScript in that same sandbox, so the loops, budget-scaled fan-out, dedup between rounds and early exits a script expressed are ordinary control flow inside an execute_code action:

Script primitive	CodeAct equivalent
await agent(prompt, opts?)	await nodetool.agents.run(prompt) (a run_subtask child)
await agent.start(prompt) + await agent.wait({ids})	await nodetool.agents.start(prompt) + await nodetool.agents.wait({ids}) — background spawn, collect later
await parallel(thunks)	await Promise.all(...), or nodetool.batch for a bound
await pipeline(items, ...stages)	nodetool.batch(items, async (item) => …)
log(message)	console.log(message)
budget	the run's own AgentPolicy bounds, enforced host-side
inputs	the step's inputs, read from context.memory via read_shared

The difference that mattered — one authored artifact reviewed before anything ran — is not lost: a plan still goes through the approval gate, and an action's code is visible in the execute_code call. What is gone is the second sandbox API, the second planner prompt, and the second set of budget knobs (maxAgentCalls).

Authoring a graph (src/author-graph.ts)

authorGraph(objective, opts) is how a graph gets written. The app-build plan stage calls it once per operation (app-build/build.ts), and the graph-planner and graph-e2e eval suites drive it directly. It streams ProcessingMessages and returns the graph.

It runs no loop of its own. runSubAgent drives one CodeAct child whose allowlist carries @nodetool-ai/sandbox-dsl, so the child authors with the typed pack — one generated function per node type — instead of the untyped node(type, props) program the retired one-shot planner took. The belt is discovery (search_nodes, get_node_info, list_nodes), validate_workflow, the Code-body harness (validate_code, run_code, test_code) and find_model when providers are configured. The graph comes back through finish() against a {nodes, edges} output schema: the object is already in the sandbox, so handing it over costs no tokens.

The prompt is assembled once. renderWorkflowAuthoringKnowledge() (prompts/workflow-authoring-knowledge.ts) is the preamble — which node to reach for, how agent steps and if_ behave, what a Code node is — and the DSL mechanics are NOT repeated there: CodeActExecutor renders GRAPH_DSL_PROMPT_SECTION itself once the pack is on the allowlist.

There is no post-hoc Code-node refinement pass. A whole-graph one-shot had no way to iterate on a Code body it wrote as a string literal, which is what such a pass existed to fix; a CodeAct child validates and re-runs a body inside the same loop. Without a catalog serving the pack the run fails immediately, naming the pack, rather than spending a provider turn on an import that cannot resolve.

Tests: tests/author-graph.test.ts (brief, preamble, belt, plus one scripted provider driving a real sandbox action over the real shipped pack).

The graph DSL program (src/graph-dsl.ts)

evaluateGraphDsl runs a free-form graph program in the QuickJS sandbox (no host access) and returns {nodes, edges}. It is the older, untyped authoring shape — node(type, properties) creates a node, passing ref.output(slot?) as a property value becomes an edge, and the program ends with return graph();:

const prompt = node("nodetool.input.StringInput", { name: "prompt" });
const image = node("nodetool.image.TextToImage", {
  prompt: prompt.output(),
  model: { provider: "fal_ai", id: "fal-ai/flux/schnell" }
});
node("nodetool.output.Output", { name: "image", value: image.output() });
return graph();

Nothing authors this way any more — authorGraph writes typed calls against @nodetool-ai/sandbox-dsl instead — but validate_workflow still accepts a program in this shape, so the evaluator and the wiring core behind it (src/graph-dsl-core.ts, shared with the pack's guest builder) stay. GraphBuilder (src/graph-builder.ts) is what loads either shape into a graph and validates it structurally, alongside node-sdk's validateGraph.

Tests: tests/graph-dsl.test.ts, tests/dsl-handle-interpolation.test.ts, tests/dsl-workflow-authoring.test.ts.

Eval suite

src/evals/ carries a provider-agnostic evaluation harness for graph authoring: GRAPH_PLANNER_EVAL_CASES (objectives + structural expectations — input wiring, node-family patterns, branch handles, reachability, prompt text, no provider-locked nodes) and runGraphPlannerEval, which drives authorGraph (metrics per case: accepted, score, authoring rounds, tool calls, duration, cost; aggregate: success rate, one-shot rate, averages). Run it against any registered provider:

Two metrics were mapped when the suite moved off the retired one-shot planner: authoring rounds replaces submit rounds and counts execute_code actions, so a one-shot is a graph delivered in the first action; attempts is gone, because authorGraph has no outer retry loop — a repair is another authoring round.

npm run dev:nodetool -- eval graph-planner --list
npm run dev:nodetool -- eval graph-planner -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval graph-planner -p ollama -m qwen-3.5:4b --cases summarize
npm run dev:nodetool -- eval graph-planner -p openai -m gpt-5.4-mini --json --out report.json
npm run dev:nodetool -- eval graph-planner -p anthropic -m ... --min-success 0.8  # floor

Where the suites run. On request. .github/workflows/agent-eval.yml (workflow_dispatch only) runs the suites that need a provider, plus jtbd, against inputs for suites/provider/model/cases and a floor override, uploads each suite's JSON, and renders one table. Nothing here gates a merge: a model run costs money and carries run-to-run variance, so it is a tool a maintainer starts after changing the planner, the CodeAct loop, the sub-agent path or a ui_* contract. The free half is a local command — nodetool eval <suite> --keyless runs the cases that need no key and no network, and --list marks them.

Harness tests (scripted provider, no network): tests/graph-planner-eval.test.ts.

End-to-end eval suite (graph-e2e)

The graph-planner suite stops at the graph: it scores structure, which says nothing about whether the workflow does what was asked. src/evals/graph-e2e- {cases,eval}.ts closes that loop — every case runs three phases and only counts as a success when all three hold:

1. plan — authorGraph() produces a graph, scored structurally by the same checkExpectations the graph-planner suite uses.
2. execute — applyRunPolicy (src/run-policy.ts) stamps the run's provider/model onto the planner's Agent nodes (the planner leaves them model-less on purpose — the run owns that choice — so an unstamped graph dies on "Select a model"). This suite is its only importer, and the reason the function outlived the retired Agent graph branch, then the graph runs for real with the case's inputs as run params, through a caller-supplied GraphRunner. The runner is injected so this package needs no execution dependency and the harness tests can drive scripted runs with no kernel; the CLI wires the real one over ExecutionSession (packages/cli/src/evals/graph-runner.ts).
3. judge — deterministic output checks (an output by name exists, is non-empty, matches/doesn't match a literal) plus an LLM judge (src/evals/goal-judge.ts) that reads the case's goal statement and the actual outputs and answers {achieved, score, reasoning} as plain JSON. A regex cannot tell a real German translation from the English echoed back; the judge can. A provider failure or unparseable answer is reported as a judge error, never as a pass.

Metrics per case: planned, executed, goalAchieved, score, submit rounds, node/ edge counts, plan and run duration, cost, plus the outputs themselves. Aggregate: end-to-end success rate (the --min-success gate), plan rate, execution rate, mean score. Cases whose graph needs a real model (needsModelProviders) skip without configured providers; the two deterministic cases (concat, arithmetic) run anywhere and use skipJudge, since their outputs are pinned exactly by pattern.

Each case costs inference twice — the run, then the judge — so it is the most expensive suite here. A full pass on claude_agent_sdk/sonnet runs ~$0.07.

npm run dev:nodetool -- eval graph-e2e --list
npm run dev:nodetool -- eval graph-e2e -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval graph-e2e -p openai -m gpt-5.4-mini --cases concat,arithmetic
npm run dev:nodetool -- eval graph-e2e -p anthropic -m ... --timeout 600000 --min-success 0.8

Harness tests (scripted provider, fake runner, no network): tests/graph-e2e-eval.test.ts.

Code node authoring eval (code-gen)

src/evals/code-gen-{cases,eval}.ts drives the real CodePlanner over eight cases — one per authoring shape the feature targets (reshape, merge/join, compute, extract/parse, split, format, validate, seed) — and scores each accepted submit_code submission structurally: declared outputs present and typed, inputs limited to the slots the dialog offered, the destination-handle case's expected output present with the right type, every declared output assigned on every visible return path (analyzeGeneratedCode), no state/ yield when nobody asked, and no name that is neither sandbox API (unknownApiReferences) nor bound by the code itself (collectBoundNames).

Acceptance is reported twice: first-pass (accepted on round 1, before the tool fed anything back) and post-repair (accepted at all within the round cap). --min-success gates on post-repair.

npm run dev:nodetool -- eval code-gen --list
npm run dev:nodetool -- eval code-gen -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval code-gen -p openai -m gpt-5.4-mini --min-success 0.9

Harness tests (scripted provider, no network): tests/code-gen-eval.test.ts.

Planning-mode eval suite (task-planner)

The graph-planner suite covers graph mode. The task planner has a suite of its own, scoring the plan statically — nothing is executed.

task-planner runs TaskPlanner.planMultiTask and scores the committed TaskPlan. PlanBuilder already rejects structurally broken plans (duplicate step ids, dangling deps, cycles), so anything that comes back is valid by construction; what it cannot judge is quality, and that is the suite: parallel width, decomposition proportional to the objective, real dependencies modelled as dependencies, tool routing (run_python for arithmetic, not a reasoning step), the step-id prefix convention, and the prompt's hard rule that final synthesis belongs to the loop that ran the plan, not to an "assemble" task. Metrics per case: tasks, steps, parallel width, critical-path depth, planner tool calls, rejected add_task/finish_plan calls; aggregate adds a clean rate — the fraction of plans built without a single rejected call.

Cases + expectations live in src/evals/task-planner-cases.ts, the runner in src/evals/task-planner-eval.ts. It offers a never-executed tool library (src/evals/planner-tools.ts: web_search, fetch_page, read_file, write_file, run_python, generate_image) so the planner has something concrete to route work to.

npm run dev:nodetool -- eval task-planner --list
npm run dev:nodetool -- eval task-planner -p anthropic -m claude-sonnet-5
IS_SANDBOX=1 npm run dev:nodetool -- eval task-planner -p claude_agent_sdk -m sonnet --no-find-model

Harness tests (scripted provider, no network): tests/task-planner-eval.test.ts.

Cost reads $0 here under claude_agent_sdk. The planner aborts the provider loop from inside the accepting tool (finish_plan), and the SDK only reports token usage on its terminal result message — which a cancelled query never emits. The run is not free; the usage is simply unobservable. Score, timing, and call counts are unaffected.

Sub-agent execution eval suite (subtask)

Where the tool-loop suites score a model driving one flat tool surface, this suite scores RunSubtaskTool — the primitive that decomposes work by spawning a child agent that inherits the parent's toolset.

The harness drives a real CodeActExecutor equipped with RunSubtaskTool plus a library of instrumented tools. Each tool records every call together with the SUBTASK_DEPTH_KEY it read from its context — 0 for a parent-level call, >= 1 for one made inside a subtask. The same tool instances serve both levels, so that field is the ground truth for who ran what, and the scorer can separate a parent that delegated from one that did the job itself.

Scoring is structural (checkSubtaskExpectations): required parent and child tools, forbidden tools, subtask-count and depth bounds, no failed subtasks, required store keys, and answer substrings — never an exact transcript, so many valid delegations pass. Metrics per case: expectation score, subtasks spawned, deepest sub-agent level, tool calls at any depth, duration, cost. The aggregate reports success rate (accepted over non-skipped), mean score, average subtasks, and total cost; --min-success gates on the success rate.

The seven cases cover one tool per delegation (delegate-compute, delegate-read-write, delegate-lookup, delegate-transform), fan-out (parallel-subtasks), a child whose tool fails so the error must surface rather than be swallowed (error-propagation), and one objective that exercises every inherited tool (all-tools).

Cases and the instrumented tool library live in src/evals/subtask-cases.ts, the runner in src/evals/subtask-eval.ts. The parent step and each subtask share a turn cap of 16 unless --max-iterations says otherwise.

npm run dev:nodetool -- eval subtask --list
npm run dev:nodetool -- eval subtask -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval subtask -p openai -m gpt-5.4-mini --cases delegate-compute,all-tools

Harness tests (scripted provider, no network): tests/subtask-eval.test.ts.

Tool-loop eval suites (frontend ui_* surfaces)

Where the graph-planner eval measures graph authoring, the tool-loop harness measures the incremental, multi-turn tool-calling flow the browser UI exposes. A real provider is handed the frontend tool contract (names/descriptions/Zod schemas mirrored from web/src/lib/tools/builtin/*) and drives it against a headless bridge — a node-side fake that holds the same state shape and applies the same mutations, with no browser. runToolLoopEval (src/evals/tool-loop-eval.ts) is generic over the surface: a case supplies a createBridge factory (HeadlessSurfaceBridge<TFinal> — { tools, finalState }) plus structural expectations, and the runner reports the same metrics as graph-planner (accepted, score, tool calls, duration, cost). Scoring is structural (checkToolLoopExpectations: required/forbidden tools, ordering, final-state predicates, tool-call budgets, no-error-results) — never an exact transcript, so many valid tool orderings pass.

Checks carry a severity, and the score weighs them by it (3/2/1 for critical/standard/advisory, scoreToolLoopChecks). Whether the required tools were called, what the final state looks like, and every escalation check are critical; ordering and no-error-results are standard; the tool-call budgets are advisory. A run that fails any critical check is additionally capped at CRITICAL_FAILURE_SCORE_CAP (0.5).

The flat pass-fraction this replaced made scores non-comparable. A live sonnet run of confirm-before-delete deleted the dead branch without ever asking — the one behavior that case exists to measure — and scored 0.62, because the graph it produced satisfied every state predicate. The same run of escalate-missing-capability escalated correctly, built the fallback the user described, and scored 0.92, docked only for exceeding a call budget. Under weighting the first is capped at 0.5 and the second lands near 0.97, which is the ordering the numbers should have had. criticalFailures per case makes it visible without reading the check list, and the text report prefixes those failures with [critical].

Severity also decides the gated metric. A case is a success only when the loop completed and no critical check failed, and successRate — what --min-success reads — counts those. "The loop ran to a stop without a provider error" is reported alongside as completionRate: it is a liveness signal, not a result, and a model that called zero tools scores 100% on it.

Eleven suites are registered:

Suite	Tools	Bridge (src/evals/)
tool-loop	ui_* graph editor	tool-loop-bridge.ts
workflow-escalation	ui_* graph editor + ask_user	tool-loop-bridge.ts + escalation.ts
script-tools	ui_script_*	surfaces/script.ts
jsscript-tools	ui_jsscript_*	surfaces/js-script.ts
sketch-tools	ui_sketch_*	surfaces/sketch.ts
timeline-tools	ui_timeline_*	surfaces/timeline.ts
storyboard-tools	ui_storyboard_*	surfaces/storyboard.ts
model3d-tools	ui_3d_*	surfaces/model3d.ts
app-tools	ui_app_* App Builder	surfaces/app.ts
memory-tools	memory_* / asset_*	surfaces/memory.ts
creative-pipeline	the three creative surfaces, composed, plus ui_brief_* / ui_review_*	surfaces/creative-pipeline.ts

creative-pipeline is the long-horizon suite: one commission carried through brief → ideation → sketch → storyboard → cut → review, scoring the seams rather than any one surface. It composes the real sketch, storyboard and timeline bridges instead of reimplementing them, so it cannot drift from the three suites that already cover those contracts, and replaces ui_storyboard_assemble_timeline with a version that actually drives the timeline bridge — the handoff is the thing under test. ui_brief_* and ui_review_* are eval instrumentation, not a frontend contract: a brief passed only in the prompt can't be told apart from one the model ignored.

Rendered clips come back 1.35× the requested length, the way a video model that emits fixed-length takes does, so a cut planned to exactly fill the brief overruns. Catching that and trimming — the last clip, since shortening an earlier one only opens a gap and leaves the runtime untouched — is what separates a scoring run from a passing-looking one.

The predicates grade outcomes, not the shape of the process. Three checks were rewritten after live runs, all the same mistake: they encoded one valid working order and failed models that used another.

• Severity was a three-value enum that threw on "critical", failing a run on this harness's vocabulary. Synonyms now map.
• Overrun detection grepped the note prose for runtime/duration/length, and scored a run that found the overrun and fixed it as a miss on wording. It now reads the severity the model assigned.
• reviewActedOn counted edits after the first review note, requiring report-before-fix. A sonnet run assembled at 16.20s, trimmed and ripple-moved to 12.00s, verified with ui_review_get_cut and then filed notes as a sign-off — a complete loop scored as "review changed nothing". It is now cutRevisedAfterAssembly, which accepts either order.

The SDK throws on its turn cap rather than stopping, so a low cap scores the whole case zero — full-pipeline needs --max-iterations 220 to clear. The suite costs real money.

IS_SANDBOX=1 npm run dev:nodetool -- eval creative-pipeline \
  -p claude_agent_sdk -m sonnet --max-iterations 220 --no-find-model

scripts/dump-creative-run.ts runs one case and writes the work itself — concepts, style-frame prompt, shot list, the assembled cut with timings, review notes, phase snapshots and the full tool transcript — to nodetool-debug/creative-<case>.{md,json}. The eval report gives pass/fail and call counts, which is right for a scoreboard and useless for seeing what the model made.

IS_SANDBOX=1 npx tsx packages/agents/scripts/dump-creative-run.ts \
  full-pipeline claude_agent_sdk sonnet 220

Live media (--live). The suite fakes every generate/render, which is what makes it a CI-priced eval. Pass --live and the same tool calls additionally hit fal, so the run leaves real stills and clips in nodetool-debug/creative-<case>-media/ without changing a tool contract or a predicate. One run's output is checked in at docs/evals/creative-pipeline/ so the suite's media can be inspected without paying for a run. MediaBackend is an interface in the bridge; the fal wiring lives in the script, because packages/agents has no fal dependency and should not grow one for an opt-in path.

Stills default to openai/gpt-image-2, clips to ltx-2-19b/distilled/image-to-video; override with CREATIVE_IMAGE_MODEL / CREATIVE_VIDEO_MODEL. The first draft picked flux/schnell on cost grounds and it was the wrong trade — flux mangles hands and the brief requires them in three of four shots. Media is the cheap part of a live run; the agent loop driving it dominates the bill.

Three caveats. The timeline still lays clips at the simulated overshoot, so the scored runtime is not the runtime of the files on disk — the real overshoot has run wider than the 1.35× modelled, so the planted defect is conservative; check a runtime assertion against the takes on disk. The provider reads FAL_API_KEY, not FAL_KEY. And no predicate can see the pixels: forbiddenAvoided reads shot text and layer names, so a run passed it while gpt-image-2 branded a bottle with lettering the brief forbade. The suite grades the plan; grading the artifact needs a human or a vision model.

FAL_API_KEY=$FAL_KEY IS_SANDBOX=1 npx tsx \
  packages/agents/scripts/dump-creative-run.ts full-pipeline claude_agent_sdk sonnet 220 --live

Permission-gated cases

A case that declares permission: { mode, approve? } runs its belt through the real gateTools ladder with a scripted approver (src/evals/tool-loop-permission.ts), and every approval request is recorded on permissionRequests for the case's expect.permissionRequests predicates, which count as final-state checks. plan-mode-blocks-mutation and denied-mutation-stays-out on the graph world assert that plan mode blocks a write without asking and that a denied write leaves the graph untouched; both are keyless. A gated case names its node types outright: ui_search_nodes has no permission class of its own, so it classifies external and plan mode blocks it.

Interactive escalation (workflow-escalation)

Every other tool-loop case is fully specified: the prompt carries everything the model needs, so guessing is never required and never penalized. This suite removes that guarantee. Each case withholds something only the user can supply — the names for an input and output, permission to delete a node, a choice between two node types that fit equally well, a capability the catalog does not have — and hands the model an ask_user tool wired to a scripted user (src/evals/escalation.ts). The question is matched against the case's reply script, the matching reply comes back as the tool result, and every exchange is recorded.

That makes the score a pair, not a single judgement: escalation.mustAsk names the reply the model has to trigger, and the case's finalState predicates check that it then built what the answer said. A model that guesses fails on the ask; one that asks the right question and ignores the reply fails on state. An off-script question gets a deliberately useless fallback answer and trips allQuestionsMatched, and askBefore is the confirm-before-you-act constraint — ui_delete_node must not precede the first ask_user.

The fifth case, no-escalation-needed, guards the opposite failure: the objective pins every value, ask_user is on the table, and reaching for it is itself the failure. Without it the suite would reward a model that asks about everything.

Escalation is a property of the generic runner, not of the graph surface — any tool-loop case on any surface can declare escalation and get the same tool and the same checks.

npm run dev:nodetool -- eval workflow-escalation --list
npm run dev:nodetool -- eval workflow-escalation -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval workflow-escalation -p openai -m gpt-5.4-mini --min-success 0.8

confirm-before-delete is the case models fail: they read the graph, delete the dead branch, and never ask. escalate-missing-capability costs the most calls, most of them ui_search_nodes hunting for an image node the catalog does not have before accepting it isn't there. Run it with --max-iterations 40 --no-find-model.

Harness tests, including a golden transcript per case so no case can be unsatisfiable: tests/escalation-tool-loop.test.ts.

memory-tools is the odd one out: instead of reimplementing a browser surface, its bridge executes the real backend tools (memory_save, memory_list, memory_search, asset_search) plus a stub generate_image against an in-memory DB (initTestDb), so it exercises the actual persistence

• resource validation a chat turn does. It scores the creative loop: generate media → remember it with an asset reference → recall it — and one case seeds a memory in a different thread, which only a cross-thread memory_search finds.

Bridges reuse the pure packages where the real logic already lives — @nodetool-ai/timeline (splitClip, ANIMATION_PRESETS, subtitle assembly, clip/track factories) — rather than reimplement. The sketch surface reimplements its layer-stack ops directly, but not its pixels: every raster layer is an @napi-rs/canvas bitmap. ui_sketch_stroke runs the editor's paint core (@nodetool-ai/image-editor/painting.js) and fill / gradient / shape / transform / adjust / crop / selection-shape run @nodetool-ai/image-editor/raster.js, both pointed at skia with setPaintSurfaceFactory(createCanvas), so a headless edit is the edit the browser would paint. ui_sketch_get_layer_image composites those layers — opacity and blend mode included, NodeTool's "normal"/"add" mapping onto Canvas's "source-over"/"lighter" — and hands the model a PNG of its own work. SketchToolBridge.compositePng() (or getLastSketchToolBridge(), for a bridge the eval runner owns) takes the finished drawing out for a human to look at. That is what makes draw-an-animal scoreable: it checks that strokes landed on several named layers and covered a real fraction of the canvas (strokedFraction, measured over stroked layers only so a solid fillColor backdrop cannot pass it) — outcomes, not a pixel-exact cat. Browser-only tools (asset capture, WebGL viewport render) are scoped out: ui_sketch_render_to_asset, ui_timeline_get_clip_frames, ui_3d_capture_view. Storyboard cannot import @nodetool-ai/llm-nodes (it depends on @nodetool-ai/agents), so its generate/render jobs are faked by flipping shot status. Its board holds wire Shot objects and normalizes every write through normalizeStoryboardScreenplay (@nodetool-ai/protocol/api-schemas), and finalState().savable reports whether storyboards.update would accept what the board now holds — a bridge that minted the id/index/status a model omits kept this suite green through a bug that lost a user's board. The app-builder surface reimplements only the Puck layout ops (nested slot tree: top-level content plus slot-valued props on Panel/Columns) headlessly — those live in web/ (puckDataOps.ts), which a backend package can't import. Its operation, variable, resource, and binding-target tools call the shared doc-ops in @nodetool-ai/app-runtime (src/doc-ops.ts), the same module the browser handler calls, so that half of the contract cannot drift. The widget types it offers come from WIDGET_CATALOG in the same package — every widget the editor ships, with the fields each accepts — so ui_app_list_component_types reports the same catalog headlessly that the browser reads off the live Puck config.

npm run dev:nodetool -- eval timeline-tools --list
npm run dev:nodetool -- eval script-tools -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval sketch-tools -p ollama -m qwen-3.5:4b --cases compose-layers
npm run dev:nodetool -- eval model3d-tools -p openai -m gpt-5.4-mini --min-success 0.8  # CI gate

Harness tests (scripted provider, no network): tests/tool-loop-eval.test.ts plus one per surface (tests/{script,js-script,sketch,timeline,storyboard,model3d,app,memory,creative-pipeline}-tool-loop.test.ts). For a live check against a real model, run the eval command above — a suite whose verdict depends on what a model chose belongs behind an explicit invocation, not in the unit suite, where a weak local model fails the run for everyone.

Running against the claude_agent_sdk provider. Two gotchas, both from the SDK's own agent loop (not the harness):

• Turn cap throws. The SDK raises error_max_turns when it reaches its turn limit, so a run that would merely stop under a stateless provider (Anthropic, Ollama) instead errors and the case scores accepted=false. Its turn accounting also counts each tool round, so the default --max-iterations 12 is easily exhausted by an over-searching model. Pass a higher cap (--max-iterations 40) when driving these suites with claude_agent_sdk.
• uid=0 refusal. The tool path runs the CLI under bypassPermissions, which it refuses as root; set IS_SANDBOX=1 (or run non-root). It must be exactly 1 — the SDK's sandbox check is value-sensitive, so an ambient IS_SANDBOX=yes (as in Claude Code on the web) does not satisfy it and the child exits with code 1 and zero tool calls, which looks like an auth/spawn failure but isn't. Override it explicitly: IS_SANDBOX=1 npm run …. See docs/AGENTS.md § Claude Agent SDK for the full nested-session recipe.

IS_SANDBOX=1 npm run dev:nodetool -- eval timeline-tools \
  -p claude_agent_sdk -m sonnet --max-iterations 40 --no-find-model

Sub-agent execution eval (subtask)

Where the tool-loop suites score a model on one flat tool surface, the subtask suite scores RunSubtaskTool — the primitive that lets an agent decompose work by spawning a fresh child agent that inherits the parent's toolset. It runs a real CodeActExecutor parent equipped with run_subtask plus six instrumented worker tools (calculate, kv_write, kv_read, lookup_fact, slugify, flaky_fail), each objective written to force delegation. The tools are shared instances at both levels; each records the SUBTASK_DEPTH_KEY it ran at, so the scorer distinguishes "the parent did it itself" (depth 0) from "the parent delegated and the child did it" (depth >= 1). Scoring is structural (checkSubtaskExpectations): required parent tools, required child tools, forbidden tools, subtask-count and depth bounds, no failed subtasks, required store keys, and answer/subtask-result substrings. Cases + tools live in src/evals/subtask-cases.ts, the runner in src/evals/subtask-eval.ts.

npm run dev:nodetool -- eval subtask --list
npm run dev:nodetool -- eval subtask -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval subtask -p openai -m gpt-5.4-mini --cases all-tools
IS_SANDBOX=1 npm run dev:nodetool -- eval subtask \
  -p claude_agent_sdk -m sonnet --max-iterations 40 --no-find-model

Its cases do not use find_model, so --no-find-model does not skip them — the primary -p provider runs both the parent and every subtask. A low score with subtasks=0 is a real finding, not a harness bug: a capable model often does trivial single-step work inline instead of delegating. Harness tests (scripted provider, no network): tests/subtask-eval.test.ts.

Mini-app build eval (app-build)

The only suite that scores a whole product loop rather than one stage: buildApp (src/app-build/) takes a prompt through spec → plan → author → check → run → judge, repairing what the oracle complains about, and the suite counts how often that ends green and how much repair it took. Cases in src/evals/app-build-cases.ts, runner in src/evals/app-build-eval.ts.

Metrics per docs/mini-app-build-harness-design.md §5.3: one-shot rate (green with zero repair rounds — the PRD's north-star number), green-within- budget rate (the suite's successRate, what --min-success gates on), repair rounds, cost, and duration.

A case is green only when the build's own verdict is ok and its target-shape checklist holds — operations, workflows, widget count, a widget nested in a container, a persist: true variable, a streaming output shown by a display widget, an operation reading a variable another wrote, and a widget carrying a condition. Without the checklist a build that shipped one operation and three widgets would score as a success. Each prompt case declares which of the six medium-complexity traits (PRD §4) it exercises; uncoveredAppBuildTraits() names any trait that lost its last case, and the harness test fails on a non-empty answer.

The two deterministic cases (greeting-card, draft-then-publish) pin the spec, bind template graphs (text transforms — no model in the app under test), author from a scripted list of ui_app_* calls, skip the judge, and assert exact widget values. They call no provider, so they run on every PR as the Quality Gate's app-build leg; what they regress is the harness, not a model. The full suite runs nightly (.github/workflows/app-build-eval.yml), reports, and gates nothing — a model's off night is not a broken build.

npm run dev:nodetool -- eval app-build --list
npm run dev:nodetool -- eval app-build -p anthropic -m claude-sonnet-5
npm run dev:nodetool -- eval app-build --cases greeting-card,draft-then-publish \
  -p ollama -m none --no-find-model --min-success 1   # no API key needed

Harness tests (scripted authoring, stub kernel runner, no network): tests/app-build-eval.test.ts.

Observing LLM Steps and Planning

Execution Tree (CLI)

The CLI renders a real-time tree view during agent execution:

✓ initialization    Starting parallel task planning...
✓ generation        Generating parallel plan...
✗ validation        Plan validation failed: duplicate step IDs
✓ generation        Retry attempt 2/3...
✓ complete          Plan created: 5 tasks, 5 steps, 5 parallelizable

◆ Plan  (3/5 tasks)
├─ ✓ Task 1: Search sources            3.2s (1/1 steps)
├─ ◐ Task 2: Analyze findings
│  ├─ ✓ google_search(query: "AI trends")
│  └─ ◐ llm_call
└─ ○ Task 3: Write report              waiting

Message Types

All execution events are yielded as ProcessingMessage:

Type	Description
planning_update	Planning phase progress (initialization, generation, validation, complete)
task_update	Task lifecycle (task_created, step_started, step_completed, step_failed, task_completed)
tool_call_update	Tool invocation with name and args
step_result	Step completion with result or error
chunk	Streaming text output
log_update	Informational log messages
llm_call	Full LLM call details (provider, model, messages, response, tokens, cost, duration)

Debug Logging

# Verbose logging to stderr
export NODETOOL_LOG_LEVEL=debug

# Log to file
export NODETOOL_LOG_FILE=/tmp/agents.log

OpenTelemetry Tracing

Span hierarchy (an analyzer agent can read this tree to optimize prompts):

workflow.run
  node.process
    agent.execute
      agent.plan        (TaskPlanner.planMultiTask / authorGraph)
        llm.chat        (BaseProvider.generateMessageTraced)
        llm.stream      (BaseProvider.generateMessagesTraced)
      agent.step        (CodeActExecutor.execute)
        llm.chat
        llm.stream

Span attributes:

• agent.*: agent.kind (execute/plan/step), agent.objective, agent.provider, agent.model, agent.tools_count, agent.task (for steps), agent.plan.kind (multi/single/graph)
• llm.*: llm.provider, llm.model, llm.request.message_count, llm.request.tools_count, llm.request.max_tokens, llm.request.stream, llm.response.content (first 2000 chars), llm.response.tool_calls_count
• gen_ai.* (OTel GenAI semconv): gen_ai.system, gen_ai.request.model, gen_ai.operation.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.total_tokens, gen_ai.usage.cost_usd
• workflow.* / node.*: workflow.id, workflow.name, workflow.node_count, node.id, node.type

Sinks (simultaneous, each on its own SpanProcessor):

# JSONL trace file — one span per line, analyzer-friendly
export NODETOOL_TRACE_FILE=/tmp/nodetool-trace.jsonl

# Stdout — pretty (human) or json (JSONL)
export NODETOOL_TRACE_STDOUT=pretty       # or "json"

# OpenTelemetry — console (legacy)
export OTEL_TRACES_EXPORTER=console
export TRACELOOP_DISABLE_BATCH=true

# OpenTelemetry — Traceloop cloud
export TRACELOOP_API_KEY=your-key

# OpenTelemetry — custom OTLP backend (Jaeger, Grafana, etc.)
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

CLI flags pass these through:

nodetool-chat --trace-file trace.jsonl
nodetool-chat --trace-stdout pretty
nodetool --trace-file trace.jsonl run workflow.ts

Telemetry must be initialized before use:

import { initTelemetry } from "@nodetool-ai/runtime";
await initTelemetry({
  traceFile: "trace.jsonl",   // optional
  stdout: "pretty",            // optional: "pretty" | "json" | false
});

The CLI calls initTelemetry() at startup automatically. The WebSocket server requires env vars to be set before starting.

Web UI

The web UI renders the same tree view in the chat panel (ExecutionTree component). The TracePanel provides a detailed event inspector with token counts, costs, and full request/response payloads.

Evaluation and Optimization

Cost Tracking

CostCalculator in @nodetool-ai/runtime tracks per-call costs based on provider pricing:

provider.trackUsage(model, { inputTokens: 100, outputTokens: 50 });
console.log(provider.getTotalCost()); // USD

Costs are logged via logProviderCall() and included in llm_call messages.

Model Selection

One run, one model. create_plan and execute_plan both use the calling session's provider and model (subAgentRuntime), so planning and step execution cannot drift apart or bill against a model the user did not pick. The per-phase model idea survives only as a possible argument on those two specs; nothing selects a second model.

Tool Result Truncation and Chat Compaction

A single result is cut, and a whole thread is summarized. Two different limits, in two different places.

• One tool result is capped at 25 000 characters (MAX_TOOL_RESULT_CHARS in constants.ts) before it enters history, with a notice telling the model the output was cut and how to ask for a smaller slice. A broad grep or a large file read otherwise returns tens of megabytes and the next request fails outright.
• A step transcript is never summarized. Step executors hand the tool-calling loop to provider.generateLoop, so each provider manages its own window — the Claude Agent SDK compacts internally, a stateless provider is sent everything. What bounds a step instead is the cumulative iteration cap and the result cut above.
• A chat thread is compacted, in packages/websocket (session/chat-compaction.ts, driven from chat-turn.ts). One summarizer call replaces everything before the last NODETOOL_CHAT_COMPACTION_KEEP_TURNS user turns, and the summary is persisted as a role: "user" row marked execution_event_type: "compaction". History assembly starts from the newest such row, so only the provider's view is shortened — the full thread stays in the database for the UI and nodetool.threads.*. The cut lands on a user message, which is what keeps a tool call attached to its result.
• Two triggers: the estimated prompt crossing NODETOOL_CHAT_COMPACTION_TOKENS before the loop, and the provider itself reporting that the prompt did not fit, which compacts and retries the turn once. A provider holding the transcript upstream skips the first, since shortening what NodeTool sends does not shorten what that provider already has.
• A failed summarizer leaves the thread uncompacted and the turn runs against the full history, with a log_update saying so. The alternative to an imperfect summary is a turn that cannot run at all.
• The user sees the cut. The web renders the compaction row as a collapsed "Earlier conversation summarized" card (web/src/components/chat/message/CompactionCard.tsx), reached from MessageView before the plain user-message path — without that branch the summary reads as something the user typed.

Plan Validation

Plans are validated before execution:

• Step/task IDs must be unique across the entire plan
• Dependencies must reference valid IDs
• No circular dependencies (DAG validation via DFS)
• On failure, the error is fed back to the model as the tool result, and it calls add_task again

Output Schema Validation

Steps can enforce structured output via JSON schema:

• additionalProperties: false enforced automatically
• Schema'd steps finalize ONLY through the finishing tool — finish_step in tool mode, finish() in a CodeAct action. There is no JSON-from-text extraction path, so a step that never calls it fails and emits an explicit error result.
• Unstructured steps (no schema) finalize when the model emits a no-tool-call assistant message; that text becomes the result.
• Two things make that contract visible to a model that believes return graph finished the step, because the observation for the losing move used to be indistinguishable from success. A CodeAct observation for a schema'd step that returned a value without finishing carries finished: false and a note — which says so explicitly when the returned value already matched the schema. And a schema'd step whose turn ended in prose is re-prompted with the contract, at most MAX_FINISH_NUDGES (2) times, before it fails.
• The failure message names the terminal state it actually hit: the provider's error, the run budget's own reason when a cost cap or deadline stopped the step, exceeded N iterations only when the loop really used its budget, else "ended after N action(s) / model turn(s) without calling finish", quoting the model's last message. A budget stop reported as "without calling finish" blamed the model for something it never got to do.
• maxIterations bounds the step, not each finish-nudge round. Each round is given what is left of the allowance and a round with nothing left is not run, so MAX_FINISH_NUDGES no longer multiplies the ceiling — a step configured for 4 model turns used to be able to make 12.

Skills System

Skills come in two tiers.

A user skill is a row in the skills table — name, description, and markdown content — that someone wrote and can rewrite.

A system skill ships with the build: a SKILL.md under packages/system-skills/<name>/, frontmatter naming it, and it is read-only at runtime. The tier exists because the two properties a user row cannot have both matter: every install has the skill on day one with no seeding migration to drift per machine, and nothing — including an agent acting on its own mis-read instructions — can edit or delete the document it is working from. system-skills.ts reads them (cached per process) from _skills/ beside the bundled server.mjs, else packages/system-skills on the way up from the module, else NODETOOL_SYSTEM_SKILLS_DIR — the same two-root shape as the sandbox packs, and for the same reason: nothing imports them, so they are not workspaces and npm links nothing. bundle-backend.mjs stages them and verify-backend-bundle.mjs fails a build that ships none.

The *-prompting skills carry a generation model line's prompting rules — image (nano-banana-pro-prompting, gpt-image-2-prompting, flux-2-klein-prompting, seedream-prompting, qwen-image-prompting), video (seedance-2-prompting, veo-3-prompting, minimax-h3-prompting, wan-2-6-prompting, kling-video-prompting, hailuo-prompting) and audio (elevenlabs-audio-prompting, stable-audio-prompting). MODEL_PROMPTING_SKILLS in model-prompting-skills.ts maps a model id onto one, and find_model attaches the answer as prompting_skill on every matching route — the catalog line is the other path in, for an agent that never called find_model. tests/model-prompting-skills.test.ts checks the table against the skills on disk, the ids the shipped provider manifests name, and the capability registry.

Both tiers share one catalog. list_skills and load_skill serve either, each answer carrying system: true|false; create_skill, update_skill and delete_skill refuse a shipped name, including a rename onto one. A user row that already held the name predates the reservation, so it wins and the shipped skill is not listed twice — reserving names stops new collisions, not old ones. A malformed SKILL.md is skipped rather than fatal: one bad shipped file must not cost a user every other skill.

Each record has name, description, and markdown content columns. Agent discovery reads the current user's records through the models layer and merges trusted sandbox-pack skills supplied for the session.

Nothing auto-selects a skill. Word overlap between an objective and a description picked the wrong document often enough that the catalog plus load_skill replaced it: the model reads what exists and asks for the one it wants. Filesystem SKILL.md discovery and the NODETOOL_AGENT_SKILL_DIRS, NODETOOL_AGENT_SKILLS and NODETOOL_AGENT_AUTO_SKILLS environment variables are gone with it.

A chat turn discovers skills the way it discovers sandbox packs — a catalog in context, a body behind a call. The websocket runner's buildSystemContent folds formatSkillCatalogForPrompt (src/skill-prompt.ts) into every turn's system prompt: one line per skill, name and description, plus the instruction to call load_skill before acting on one. The block is ephemeral — it reaches the provider and is never persisted, so it reflects the table as it is right now.

/<name> skips that round trip. A message naming a skill with a leading slash is asking for it, so findInvokedSkillNames matches the text against the user's own names and formatInvokedSkillsForPrompt puts those bodies in the same block. The match needs a word boundary, so src/utils is a path and not an invocation. The composer's / autocomplete (web/src/components/chat/composer/useTextareaSkillMention.tsx) only types the text; the expansion is server-side, so a headless client gets it too.

The skills capability module (src/capabilities/skills.ts) is the rest:

Capability	Does
list_skills	Names and descriptions, optionally filtered by query
load_skill	One skill's full instructions by name
create_skill	Author a new skill (names are unique per user)
update_skill	Change name, description or body
delete_skill	Remove one

Every one is scoped to context.userId and refuses a run that carries none. Tests: tests/capabilities-skills.test.ts.

Tuning Checklist

1. Reduce cost: run the session on a cheaper model, and give the run a RunBudget with a USD cap so a loop stops instead of billing
2. Improve plan quality: pass a systemPrompt preamble to TaskPlanner — it is prepended to the TaskArchitect contract, not swapped for it
3. Speed up execution: decompose into more independent tasks, since dependency edges are what serialize the DAG
4. Control scope: maxStepIterations bounds one step's action rounds; the run's RunBudget bounds everything else — nothing counts dispatch rounds
5. Validate output: use outputSchema to enforce structured step results
6. Restrict tools: per-step tools arrays limit which tools a step can call
7. Observe: enable tracing (OTEL_TRACES_EXPORTER=console) to see every LLM call
8. Iterate on skills: add domain-specific records in the Skills panel, and name one with /<name> to load its body without a round trip

Authoring Agent Nodes — Pitfalls

When building a node that wraps an agent (e.g. llm-nodes AgentNode):

• Every tool named in an agent's system prompt must actually be registered in its toolset. A prompt-referenced-but-unregistered tool is a silent no-op. Resolve real builtin tools (resolveBuiltinAgentTool) and don't reference tools you didn't wire.
• Every declared prop must be consumed by process() or injected into the prompt. A declared-but-unwired prop (max_output_chars, url, output_dir) does nothing.
• yield structured results so the kernel routes them to dynamic output handles; don't return them from a generator (yield* discards the return value). Keep structured-output emission consistent across modes (loop vs plan).