Skip to content

code-gen: fix the four functional issues, then drop the rollout flag - #4629

Merged
georgi merged 3 commits into
mainfrom
claude/code-gen-functional-issues-aqxo6a
Aug 1, 2026
Merged

code-gen: fix the four functional issues, then drop the rollout flag#4629
georgi merged 3 commits into
mainfrom
claude/code-gen-functional-issues-aqxo6a

Conversation

@georgi

@georgi georgi commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #4614. Four issues survived that PR's green CI, because nothing exercises a generation end to end. All four are fixed here, and the rollout flag is then removed.

P1 — Seeded handle contracts are now enforced

The two handle entry points create the edge before the model runs, so the submission has to produce that handle. SubmitCodeTool only checked the submission against itself, so a renamed or dropped port left the pre-created edge attached to a handle the node no longer had — a broken graph that passes every self-consistency check the transport schema can make.

CodePlanner now passes the connected ports into the tool, which rejects a submission that renames, removes, or incompatibly retypes one, as a feedback round the model can fix. portTypesCompatible is deliberately lenient — it catches a seeded list coming back as str, and stays out of the way for custom node types, unions, and widened containers. It is also symmetric by construction, which is what lets the call sites read in wire order; the doc comment says so and a test pins it. The user prompt states the constraint too, so it usually does not come up.

P1 — "Ask AI" no longer strands edges

An accepted submission replaces a node's dynamic inputs and outputs wholesale, while applyCodeGenSubmission touches only node data, so any edge on a dropped handle was left dangling. Until generation can remap edges, the button is hidden on a Code node that already has code or any connected handle. The handle entry points are unaffected — their seeded port is now guaranteed to survive.

P2 — Static imports no longer pass validation

The analyzer parsed the body as a top-level ES module, so import x from "pkg"; return { x }; was accepted and then failed at run time, where the same text is spliced into an async function. Module declarations are rejected explicitly, with the reason the sandbox has no loader. await import(...) is a call expression and still reaches the sandbox.

P1 → the flag is gone

The third P1 was that NODETOOL_CODE_GENERATION hid the UI without disabling generation server-side: an authenticated caller, or any local-mode client, could spend provider credits on a deployment that had the feature off.

The flag existed to hold AI authoring back until it worked, and with the issues above fixed there is nothing left for it to gate — so rather than add the missing server-side check, both halves are removed. A gate nobody sets is a second code path that only rots, which is exactly how the UI and the endpoint drifted apart in the first place.

Removed: NODETOOL_CODE_GENERATION, isCodeGenerationEnabled() on server and client, the codeGeneration field on /api/config and its mirror in the web runtime config, the disabled failure code and its dialog copy, and the flag checks in front of the palette entry, both handle context menus, and the Ask AI button. What still gates Ask AI is a property of the node, not of the deployment.

Testing

  • packages/agents/tests/code-gen-seeded-ports.test.ts (new) — rename, drop, and retype rejected for both a seeded input and an expected output; added inputs still allowed; no contract when nothing was seeded; portTypesCompatible unit cases including symmetry.
  • code-planner.test.ts — a renamed seeded input is rejected and then fixed over a feedback round; every-round failure surfaces as no_valid_submission naming the port.
  • code-gen-analyze.test.ts — static import and top-level export rejected; await import(...) still accepted.
  • CodeBody.test.tsx — Ask AI hidden on a node with code, hidden on a connected node, kept when the edges belong to other nodes.
  • Flag-specific tests removed across config-api, trpc-code-gen, runtimeConfig, useMetadata, CodeGenMenuItems and CodeBody; the palette-entry test now asserts the entry is always present.

lint, typecheck (web + electron), the web (12,486) and electron (659) suites, and test:packages all pass. Two unrelated pre-existing sandbox failures: image-nodes needs a Vulkan driver, and the mobile typecheck leg needs Expo deps installed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6

claude added 2 commits August 1, 2026 15:58
Four issues survived green CI on the merged revision, because nothing
exercises a generation end to end.

Seeded handle contracts are now enforced. The two handle entry points
create the edge before the model runs, so the submission has to produce
that handle — but SubmitCodeTool only checked the submission against
itself, and a renamed or dropped port left the edge attached to a handle
the node no longer had. The planner passes the connected ports into the
tool, which rejects a submission that renames, removes, or incompatibly
retypes one, as a feedback round the model can fix. The user prompt now
states the constraint too, so it usually does not come up.

The rollout flag gates the server, not just the UI. codeGen.generate
never read isCodeGenerationEnabled(), so an authenticated caller or any
local-mode client could spend provider credits with NODETOOL_CODE_GENERATION
off. Checked ahead of the in-flight slot and the provider, and reported
as a new `disabled` failure the dialog words for itself.

Ask AI no longer strands edges. An accepted submission replaces a node's
dynamic inputs and outputs wholesale while applyCodeGenSubmission touches
only node data, so any edge on a dropped handle was left dangling. Until
generation can remap edges, the button is hidden on a Code node that
already has code or any connected handle. The handle entry points are
unaffected — their seeded port is now guaranteed to survive.

Static imports no longer pass validation. The analyzer parsed the body as
a top-level module, so `import x from "pkg"; return { x };` was accepted
and then failed at run time, where the same text is spliced into an async
function. Module declarations are rejected explicitly, with the reason the
sandbox has no loader; `await import(...)` is a call expression and still
reaches the sandbox.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
The three P1 issues the flag was holding the feature back for are fixed,
so the default flips: code generation is on unless a deployment sets
NODETOOL_CODE_GENERATION to 0/false/off/no. An operator who does not want
their users spending provider credits this way still has one variable to
set; everyone else gets the feature without one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
Copilot AI review requested due to automatic review settings August 1, 2026 16:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Code-node AI authoring by enforcing seeded handle contracts, moving the rollout gate to the server-side mutation, preventing edge-stranding in the UI entry point, and rejecting generated code that can’t run in the runtime wrapper. It also inverts the NODETOOL_CODE_GENERATION default to “on unless opted out,” enabling rollout by default.

Changes:

  • Enforce “seeded ports must survive” by passing required/expected ports into SubmitCodeTool and validating name/type compatibility.
  • Gate generation server-side (tRPC) and surface a dedicated disabled failure code/UI copy; invert rollout flag default to enabled unless explicitly opted out.
  • Prevent edge-stranding by hiding “Ask AI” on Code nodes that already have code or are connected; reject top-level import/export in generated code analysis.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
web/src/components/node_types/CodeBody.tsx Hide “Ask AI” when Code node has existing code or connected edges to avoid stranded handles.
web/src/components/node_types/code_gen/CodeGenDialog.tsx Add UI copy for new disabled server-side gate failure.
web/src/components/node_types/tests/CodeBody.test.tsx Add coverage for “Ask AI” visibility rules (existing code, connected edges, unrelated edges).
packages/websocket/tests/trpc-code-gen.test.ts Add test ensuring opted-out deployments return disabled before slot/provider resolution; isolate env var.
packages/websocket/tests/config-api.test.ts Update config endpoint expectations for inverted default + opt-out.
packages/websocket/src/trpc/routers/code-gen.ts Enforce server-side isCodeGenerationEnabled() gate and return disabled error.
packages/protocol/tests/api-schemas-code-gen.test.ts Add schema variant coverage for disabled error.
packages/protocol/src/api-schemas/code-gen.ts Add disabled to error codes + discriminated union.
packages/config/src/code-generation.ts Invert NODETOOL_CODE_GENERATION default: on unless explicitly off (0/false/off/no).
packages/config/src/tests/code-generation.test.ts Update tests for inverted default and falsy spellings.
packages/agents/tests/code-planner.test.ts Add feedback-round coverage for rejecting renamed seeded input and surfacing persistent failure.
packages/agents/tests/code-gen-seeded-ports.test.ts New tests for seeded input / expected output enforcement + portTypesCompatible cases.
packages/agents/tests/code-gen-analyze.test.ts Add tests rejecting top-level import/export and allowing dynamic import expression.
packages/agents/src/tools/submit-code-tool.ts Enforce seeded port contracts in-tool (required inputs + expected output) after alias validation.
packages/agents/src/code-planner.ts Pass seeded required inputs / expected output into SubmitCodeTool.
packages/agents/src/code-gen/prompt.ts Add explicit prompt constraint requiring seeded inputs be declared exactly.
packages/agents/src/code-gen/port-types.ts Add portTypesCompatible + formatPortType to support seeded contract checks with lenient compatibility.
packages/agents/src/code-gen/analyze.ts Reject module declarations (import/export) at top level to match runtime wrapper constraints.

`and must stay in the submission under that exact name. Declare it as ` +
`"${expectedOutput.name}": ${formatPortType(expectedOutput.type)}.`
);
} else if (!portTypesCompatible(match.type, expectedOutput.type)) {
The flag existed to hold AI authoring back until it worked. With the four
functional issues fixed there is nothing left for it to gate, and a gate
nobody sets is a second code path that only ever rots — the last revision
proved it, by hiding the UI while leaving the endpoint open.

Gone: NODETOOL_CODE_GENERATION, isCodeGenerationEnabled() on both the
server and the client, the `codeGeneration` field on /api/config and its
mirror in the web runtime config, the `disabled` failure code and its
dialog copy, and the flag checks in front of the palette entry, the two
handle context menus and the Ask AI button.

What still gates Ask AI is a property of the node, not of the deployment:
it stays hidden on a Code node that already has code or a connected
handle, because an accepted submission replaces the node's handles
wholesale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GQqPtQhkzQxnzPuGPswXn6
Copilot AI review requested due to automatic review settings August 1, 2026 16:27
@georgi georgi changed the title code-gen: fix the four functional issues, then turn AI authoring on code-gen: fix the four functional issues, then drop the rollout flag Aug 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

@georgi
georgi enabled auto-merge (squash) August 1, 2026 16:34
@georgi
georgi merged commit 2eec871 into main Aug 1, 2026
25 checks passed
@georgi
georgi deleted the claude/code-gen-functional-issues-aqxo6a branch August 1, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants