code-gen: fix the four functional issues, then drop the rollout flag - #4629
Merged
Conversation
Four issues survived green CI on the merged revision, because nothing
exercises a generation end to end.
Seeded handle contracts are now enforced. The two handle entry points
create the edge before the model runs, so the submission has to produce
that handle — but SubmitCodeTool only checked the submission against
itself, and a renamed or dropped port left the edge attached to a handle
the node no longer had. The planner passes the connected ports into the
tool, which rejects a submission that renames, removes, or incompatibly
retypes one, as a feedback round the model can fix. The user prompt now
states the constraint too, so it usually does not come up.
The rollout flag gates the server, not just the UI. codeGen.generate
never read isCodeGenerationEnabled(), so an authenticated caller or any
local-mode client could spend provider credits with NODETOOL_CODE_GENERATION
off. Checked ahead of the in-flight slot and the provider, and reported
as a new `disabled` failure the dialog words for itself.
Ask AI no longer strands edges. An accepted submission replaces a node's
dynamic inputs and outputs wholesale while applyCodeGenSubmission touches
only node data, so any edge on a dropped handle was left dangling. Until
generation can remap edges, the button is hidden on a Code node that
already has code or any connected handle. The handle entry points are
unaffected — their seeded port is now guaranteed to survive.
Static imports no longer pass validation. The analyzer parsed the body as
a top-level module, so `import x from "pkg"; return { x };` was accepted
and then failed at run time, where the same text is spliced into an async
function. Module declarations are rejected explicitly, with the reason the
sandbox has no loader; `await import(...)` is a call expression and still
reaches the sandbox.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
The three P1 issues the flag was holding the feature back for are fixed, so the default flips: code generation is on unless a deployment sets NODETOOL_CODE_GENERATION to 0/false/off/no. An operator who does not want their users spending provider credits this way still has one variable to set; everyone else gets the feature without one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens Code-node AI authoring by enforcing seeded handle contracts, moving the rollout gate to the server-side mutation, preventing edge-stranding in the UI entry point, and rejecting generated code that can’t run in the runtime wrapper. It also inverts the NODETOOL_CODE_GENERATION default to “on unless opted out,” enabling rollout by default.
Changes:
- Enforce “seeded ports must survive” by passing required/expected ports into
SubmitCodeTooland validating name/type compatibility. - Gate generation server-side (tRPC) and surface a dedicated
disabledfailure code/UI copy; invert rollout flag default to enabled unless explicitly opted out. - Prevent edge-stranding by hiding “Ask AI” on Code nodes that already have code or are connected; reject top-level
import/exportin generated code analysis.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| web/src/components/node_types/CodeBody.tsx | Hide “Ask AI” when Code node has existing code or connected edges to avoid stranded handles. |
| web/src/components/node_types/code_gen/CodeGenDialog.tsx | Add UI copy for new disabled server-side gate failure. |
| web/src/components/node_types/tests/CodeBody.test.tsx | Add coverage for “Ask AI” visibility rules (existing code, connected edges, unrelated edges). |
| packages/websocket/tests/trpc-code-gen.test.ts | Add test ensuring opted-out deployments return disabled before slot/provider resolution; isolate env var. |
| packages/websocket/tests/config-api.test.ts | Update config endpoint expectations for inverted default + opt-out. |
| packages/websocket/src/trpc/routers/code-gen.ts | Enforce server-side isCodeGenerationEnabled() gate and return disabled error. |
| packages/protocol/tests/api-schemas-code-gen.test.ts | Add schema variant coverage for disabled error. |
| packages/protocol/src/api-schemas/code-gen.ts | Add disabled to error codes + discriminated union. |
| packages/config/src/code-generation.ts | Invert NODETOOL_CODE_GENERATION default: on unless explicitly off (0/false/off/no). |
| packages/config/src/tests/code-generation.test.ts | Update tests for inverted default and falsy spellings. |
| packages/agents/tests/code-planner.test.ts | Add feedback-round coverage for rejecting renamed seeded input and surfacing persistent failure. |
| packages/agents/tests/code-gen-seeded-ports.test.ts | New tests for seeded input / expected output enforcement + portTypesCompatible cases. |
| packages/agents/tests/code-gen-analyze.test.ts | Add tests rejecting top-level import/export and allowing dynamic import expression. |
| packages/agents/src/tools/submit-code-tool.ts | Enforce seeded port contracts in-tool (required inputs + expected output) after alias validation. |
| packages/agents/src/code-planner.ts | Pass seeded required inputs / expected output into SubmitCodeTool. |
| packages/agents/src/code-gen/prompt.ts | Add explicit prompt constraint requiring seeded inputs be declared exactly. |
| packages/agents/src/code-gen/port-types.ts | Add portTypesCompatible + formatPortType to support seeded contract checks with lenient compatibility. |
| packages/agents/src/code-gen/analyze.ts | Reject module declarations (import/export) at top level to match runtime wrapper constraints. |
| `and must stay in the submission under that exact name. Declare it as ` + | ||
| `"${expectedOutput.name}": ${formatPortType(expectedOutput.type)}.` | ||
| ); | ||
| } else if (!portTypesCompatible(match.type, expectedOutput.type)) { |
The flag existed to hold AI authoring back until it worked. With the four functional issues fixed there is nothing left for it to gate, and a gate nobody sets is a second code path that only ever rots — the last revision proved it, by hiding the UI while leaving the endpoint open. Gone: NODETOOL_CODE_GENERATION, isCodeGenerationEnabled() on both the server and the client, the `codeGeneration` field on /api/config and its mirror in the web runtime config, the `disabled` failure code and its dialog copy, and the flag checks in front of the palette entry, the two handle context menus and the Ask AI button. What still gates Ask AI is a property of the node, not of the deployment: it stays hidden on a Code node that already has code or a connected handle, because an accepted submission replaces the node's handles wholesale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #4614. Four issues survived that PR's green CI, because nothing exercises a generation end to end. All four are fixed here, and the rollout flag is then removed.
P1 — Seeded handle contracts are now enforced
The two handle entry points create the edge before the model runs, so the submission has to produce that handle.
SubmitCodeToolonly checked the submission against itself, so a renamed or dropped port left the pre-created edge attached to a handle the node no longer had — a broken graph that passes every self-consistency check the transport schema can make.CodePlannernow passes the connected ports into the tool, which rejects a submission that renames, removes, or incompatibly retypes one, as a feedback round the model can fix.portTypesCompatibleis deliberately lenient — it catches a seededlistcoming back asstr, and stays out of the way for custom node types, unions, and widened containers. It is also symmetric by construction, which is what lets the call sites read in wire order; the doc comment says so and a test pins it. The user prompt states the constraint too, so it usually does not come up.P1 — "Ask AI" no longer strands edges
An accepted submission replaces a node's dynamic inputs and outputs wholesale, while
applyCodeGenSubmissiontouches only node data, so any edge on a dropped handle was left dangling. Until generation can remap edges, the button is hidden on a Code node that already has code or any connected handle. The handle entry points are unaffected — their seeded port is now guaranteed to survive.P2 — Static imports no longer pass validation
The analyzer parsed the body as a top-level ES module, so
import x from "pkg"; return { x };was accepted and then failed at run time, where the same text is spliced into an async function. Module declarations are rejected explicitly, with the reason the sandbox has no loader.await import(...)is a call expression and still reaches the sandbox.P1 → the flag is gone
The third P1 was that
NODETOOL_CODE_GENERATIONhid the UI without disabling generation server-side: an authenticated caller, or any local-mode client, could spend provider credits on a deployment that had the feature off.The flag existed to hold AI authoring back until it worked, and with the issues above fixed there is nothing left for it to gate — so rather than add the missing server-side check, both halves are removed. A gate nobody sets is a second code path that only rots, which is exactly how the UI and the endpoint drifted apart in the first place.
Removed:
NODETOOL_CODE_GENERATION,isCodeGenerationEnabled()on server and client, thecodeGenerationfield on/api/configand its mirror in the web runtime config, thedisabledfailure code and its dialog copy, and the flag checks in front of the palette entry, both handle context menus, and the Ask AI button. What still gates Ask AI is a property of the node, not of the deployment.Testing
packages/agents/tests/code-gen-seeded-ports.test.ts(new) — rename, drop, and retype rejected for both a seeded input and an expected output; added inputs still allowed; no contract when nothing was seeded;portTypesCompatibleunit cases including symmetry.code-planner.test.ts— a renamed seeded input is rejected and then fixed over a feedback round; every-round failure surfaces asno_valid_submissionnaming the port.code-gen-analyze.test.ts— staticimportand top-levelexportrejected;await import(...)still accepted.CodeBody.test.tsx— Ask AI hidden on a node with code, hidden on a connected node, kept when the edges belong to other nodes.config-api,trpc-code-gen,runtimeConfig,useMetadata,CodeGenMenuItemsandCodeBody; the palette-entry test now asserts the entry is always present.lint,typecheck(web + electron), the web (12,486) and electron (659) suites, andtest:packagesall pass. Two unrelated pre-existing sandbox failures:image-nodesneeds a Vulkan driver, and the mobile typecheck leg needs Expo deps installed.🤖 Generated with Claude Code
https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6