Skip to content

fix(agents): close the failure modes a workflow-authoring session hit - #5330

Merged
georgi merged 1 commit into
mainfrom
claude/competitor-ad-hook-workflow-bjnuln
Aug 28, 2026
Merged

fix(agents): close the failure modes a workflow-authoring session hit#5330
georgi merged 1 commit into
mainfrom
claude/competitor-ad-hook-workflow-bjnuln

Conversation

@georgi

@georgi georgi commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

What changed

A chat session asked for a competitor-ad workflow and never got one — twelve authoring rounds, no graph. Every wall it hit is a fixable product failure, and this closes all eight. In order of how much churn each one cost:

search_nodes scored characters. The schema says string[], and a model that sent the bare string "serpapi" had it accepted: a string is iterable, so the scorer walked it one character at a time and every node whose text held an "s", an "e" or an "r" matched. The live registry answered that query with 322 results led by CompareImages at score 133 — noise a caller cannot tell from a ranked answer, and no signal that it passed the wrong shape. A lone string is one term now; anything else is refused by name. A search that matches nothing also says which of the two default-on filters could be hiding the answer, instead of returning an empty list.

list_nodes on an unknown namespace was a dead end. nodetool.web answered {total: 0, namespaces: {}, nodes: []}, which reads exactly like a namespace that exists and is empty. It now names the miss and hands back the namespaces that do exist, nearest first.

find_model(...).ref was undefined. Every caller wants one model, and results[0].ref reads as if results and ref were alternatives. undefined is not an error, so it went into a node's model property and surfaced much later as "Property model requires a language_model to be selected" on a graph whose model line looked right. The answer carries the best model's ref at the top level, and the validator's hint points at it.

Interpolating a handle into a string had no way out. "Pass it as the property value itself" answers the one-handle case and nothing else; a prompt built from four upstream values has nowhere to go, and the session rewrote the same template eight times before giving up. Both DSL cores now name nodetool.text.Template and show the call.

A misspelled pack export reached the guest. websearch for webSearch came back as QuickJS's Could not find export, naming neither the module's exports nor the near miss. Pack modules get the admission-time pre-flight capability modules already had: exportedNames parses the resolved source, and the refusal carries the export list plus the suggestion. A module whose exports cannot be decided (export * from) is left alone — an unknowable list must not read as an empty one. require() and dynamic import() are refused there too, instead of ReferenceError: require is not defined.

console.log(fn) killed the run. The wrapper marshals an argument property by property and a function's prototype chain leads back to itself, so logging one — or a module namespace, which is what import * as ns gives — blew the host stack, left QuickJS handles alive, and aborted the runtime with list_empty(&rt->gc_obj_list), after the guest's real work was done. Arguments are formatted guest-side now, so only strings cross: a function prints as [Function: name], a cycle as [Circular], everything else byte-identically to before.

Eight bullets for one problem. nodetool.workflows.validate threw the first eight issues of any severity, burying one real error under seven "untyped dynamic slot" info notes — and a model reading eight bullets rewrites eight things. report.ok is decided by the errors alone, so the throw lists the errors alone and counts the rest.

Syntax errors pointed at the wrong line. QuickJS compiles the whole entry module, so import { x } = "…" on line 3 arrived as expecting '(' at action:1:26 with an excerpt showing a perfectly good line; the model rewrote line 1 twice. A body acorn also rejects is re-diagnosed with acorn's position, so this can never override a failure the parser is happy with.

One adjacent fix the diff needed: the capability gate's suites are file paths, and they do not move when a case is added to a file already listed — which is what covering a new contract usually looks like. A description edit therefore demanded a contrived change to a generated field, while writing the actual test satisfied nothing. A contract change whose own covering suite is in the diff is now answered.

Verification

  • npm run test:affected — its plan is "everything", because a packages/sandbox-packs/ file belongs to no workspace. Ran per-suite instead; every touched package and every direct dependent is green:

    Workspace Result
    agents 226 files, 3460 passed
    node-sdk 69 files, 1190 passed
    base-nodes 33 files, 2027 passed
    kernel 70 files, 983 passed
    cli 56 files, 671 passed
    execution 34 files, 293 passed
    code-nodes 12 files, 207 passed
    sandbox-compiler 9 files, 208 passed (via the gate)
  • npm run typecheck

  • npm run lint (exit 0; the warnings it prints are pre-existing and in files this diff does not touch)

  • npm run dev:nodetool -- harness gate --base main6/6 selfchecks passed

  • npm run build:sandbox-dsl:check — the hand-written core.js edit leaves the 70 generated modules unchanged

Three environment gaps in the container, all predating this branch and none touched by the diff. base-nodes first failed 14 tests on No WebGPU adapter available (Node/Dawn); extracting mesa-vulkan-drivers and pointing VK_DRIVER_FILES at lavapipe — the recipe AGENTS.md documents, and what CI installs for that leg — took it to 33/33. @nodetool-ai/model3d had no node_modules link, so build:packages could not finish until I symlinked it. And packages/protocol fails two suites on Cannot find package 'ajv/dist/2020.js': the hoisted ajv is v6 (a transitive dep) while packages/protocol declares ^8. git stash reproduces all three.

Every new check was inverted once and observed failing:

Inverted Test that went red
searchTerms → the old params["query"] as string[] reads a lone string query as one term, not as its characters, refuses a query that is neither a string nor an array
unknownPackExports removed refuses an export the pack module does not have, with the list
dynamicModuleAccess guard disabled refuses require() and dynamic import(), naming the static form
severity filter → the old "any issue" lists the errors and counts the rest
reparseSyntaxError removed reports acorn's position, not QuickJS's
top-level answer.ref removed answers the best model's ref at the top level
the template hint removed from the pack's core.js refuses concatenation, and the message says what to do instead (through the real QuickJS guest)

The console fix was observed the other way round: before it, console.log('fn:', function f(){}) aborted the runtime with Aborted(Assertion failed: list_empty(&rt->gc_obj_list), at: ../../vendor/quickjs-ng/quickjs.c,1954,JS_FreeRuntime).

Agent capabilities

  • npm run capabilities:check passes (npm run capabilities:sync regenerated the three moved contract fingerprints)
  • find_model, list_nodes and search_nodes re-declare their contract; each names packages/agents/tests/capabilities-{models,nodes}.test.ts, and this diff adds the covering cases to those files

New checks

  • Each was inverted once and observed failing — the table above
  • The pack-export pre-flight cannot pass by matching nothing: leaves a module whose exports cannot be read alone pins that an undecidable export list means permission, and exportedNames has its own suite asserting it finds the names in each export form

🤖 Generated with Claude Code

https://claude.ai/code/session_01CsXu7kKHebJAeKPhyDZ18v

A session asked for a competitor-ad workflow and never got one. Twelve
rounds, no graph. Every wall it hit is fixed here.

**search_nodes scored characters.** The schema says `string[]`, and a
model that sent the bare string `"serpapi"` had it accepted: a string is
iterable, so the scorer walked it one character at a time and every node
holding an "s", an "e" or an "r" matched. The live registry answered with
322 results led by `CompareImages` at score 133 — noise the caller cannot
tell from a ranked answer. A lone string is one term now; anything else is
refused by name.

**list_nodes on an unknown namespace was a dead end.** `nodetool.web`
answered `{total: 0, namespaces: {}, nodes: []}`, which reads the same as a
namespace that exists and is empty. It now names the miss and hands back
the namespaces that do exist. A search that matches nothing likewise says
what the two default filters hide, instead of an empty list.

**find_model(...).ref was undefined.** Every caller wants one model, and
`results[0].ref` reads as if `results` and `ref` were alternatives. The
`undefined` went into a node's model property and surfaced much later as
"Property model requires a language_model to be selected" on a graph whose
model line looked right. The answer carries the best model's `ref` at the
top level, and the validator's hint says so.

**Interpolating a handle into a string had no way out.** "Pass it as the
property value itself" answers the one-handle case; a prompt built from
four upstream values has nowhere to go, and the session rewrote the same
template eight times. Both DSL cores now name `nodetool.text.Template` and
show the call.

**A misspelled pack export reached the guest.** `websearch` for `webSearch`
came back as QuickJS's "Could not find export", naming neither the module's
exports nor the near miss. Pack modules get the pre-flight capability
modules already had: `exportedNames` reads the resolved source, and the
refusal carries the list and the suggestion. `require()` and dynamic
`import()` are refused there too, instead of `ReferenceError: require is
not defined`.

**console.log(fn) killed the run.** The wrapper marshals an argument
property by property and a function's prototype chain leads back to itself,
so logging one — or a module namespace, which is what `import * as ns`
gives — blew the host stack, left handles alive and aborted the runtime with
`list_empty(&rt->gc_obj_list)`, after the guest's real work was done.
Arguments are formatted guest-side now, so only strings cross.

**Eight bullets for one problem.** `nodetool.workflows.validate` threw the
first eight issues of any severity, burying one error under seven "untyped
dynamic slot" info notes. It throws the errors, and counts the rest.

**Syntax errors pointed at the wrong line.** QuickJS compiles the whole
entry module and reported `import { x } = "…"` on line 3 as `expecting '('
at action:1:26`, with an excerpt showing a good line; the model rewrote line
1 twice. A body acorn also rejects is re-diagnosed with acorn's position.

The capability gate is fixed alongside: `suites` are file paths, so they do
not move when a case is added to a file already listed, and a description
edit demanded a contrived change to a generated field. A contract change
whose own covering suite is in the diff is answered.
@georgi
georgi enabled auto-merge August 28, 2026 12:55
@georgi
georgi merged commit 3751123 into main Aug 28, 2026
22 checks passed
@georgi
georgi deleted the claude/competitor-ad-hook-workflow-bjnuln branch August 28, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants