-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat: run code-mode MCP in Vercel Sandbox #2626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shrey150
wants to merge
31
commits into
shrey/stg-2765-codemode-publish
Choose a base branch
from
shrey/stg-2765-codemode-vercel
base: shrey/stg-2765-codemode-publish
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
5b207ec
feat(vercel): add Stagehand code-mode MCP example
shrey150 85e9ac3
test(integrations): smoke Vercel code mode in CI
shrey150 95902d6
Merge updated code-mode guidance parent
shrey150 eca28f1
Merge updated code-mode guidance parent
shrey150 fd2003a
Merge updated code-mode guidance parent
shrey150 175128f
Merge refreshed code-mode guidance parent
shrey150 f95a3bd
fix(vercel): use current agent instructions field
shrey150 d41e0f9
feat: add sandboxed code-mode image
shrey150 a434b4c
Merge remote-tracking branch 'origin/shrey/stg-2765-codemode-sandbox'…
shrey150 bd2c800
feat(vercel): run code mode in an E2B sandbox
shrey150 e4bbe74
fix: cover tsconfig changes in image CI
shrey150 c43ed42
Merge sandbox foundation updates
shrey150 df3959d
feat: add a source-installed E2B MCP boundary
shrey150 a804d84
fix: make E2B egress default deny
shrey150 8155c03
fix: keep MCP session setup offline
shrey150 8cf5b40
fix: poll E2B build readiness safely
shrey150 3a54a90
fix: terminate the offline setup branch
shrey150 bc92188
fix: validate the offline source mirror
shrey150 f8c6e80
docs: bound E2B custom servers to one tool call
shrey150 a6c7954
feat: run code-mode MCP in Vercel Sandbox
shrey150 ec7a7ee
fix: use the locked sandbox gateway
shrey150 e9c6682
fix(vercel): harden sandbox lifecycle boundary
shrey150 938efe1
test(vercel): make sandbox contracts deterministic
shrey150 48b87ba
Merge remote-tracking branch 'origin/shrey/stg-2765-codemode-publish'…
shrey150 4678772
refactor(vercel): install exact code-mode package artifacts
shrey150 245c2bb
test(vercel): cover invalid proxy port configuration
shrey150 75aa26c
fix(vercel): harden package-installed sandbox setup
shrey150 6a01338
fix(ci): register sandbox build task
shrey150 63ddf58
fix(vercel): close remaining setup review gaps
shrey150 35c1578
test(vercel): lock credential and pack failure branches
shrey150 e3e70f2
test(vercel): make stderr sentinel deterministic
shrey150 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| name: Code-mode framework examples | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - ".github/workflows/codemode-framework-examples.yml" | ||
| - "packages/integrations/**" | ||
| - "packages/extension/**" | ||
| - "packages/protocol/**" | ||
| - "packages/sdk-ts/**" | ||
| - "pnpm-lock.yaml" | ||
| - "pnpm-workspace.yaml" | ||
| - "turbo.json" | ||
| push: | ||
| branches: [main, v4-spike] | ||
| paths: | ||
| - ".github/workflows/codemode-framework-examples.yml" | ||
| - "packages/integrations/**" | ||
| - "packages/extension/**" | ||
| - "packages/protocol/**" | ||
| - "packages/sdk-ts/**" | ||
| - "pnpm-lock.yaml" | ||
| - "pnpm-workspace.yaml" | ||
| - "turbo.json" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| vercel: | ||
| name: Vercel AI SDK | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| steps: | ||
| - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 | ||
|
|
||
| - uses: ./.github/actions/setup-node-pnpm | ||
| with: | ||
| use-prebuilt-artifacts: "false" | ||
|
|
||
| - uses: ./.github/actions/setup-chrome-verified | ||
| id: setup-chrome | ||
|
|
||
| - run: pnpm exec turbo run build --filter @browserbasehq/stagehand-integrations | ||
| - run: pnpm --filter @browserbasehq/stagehand-integrations-example-vercel typecheck | ||
| - run: pnpm --filter @browserbasehq/stagehand-integrations-example-vercel smoke | ||
| env: | ||
| CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }} | ||
| STAGEHAND_BROWSER: local | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Vercel AI SDK + Stagehand code mode | ||
|
|
||
| This example connects the Vercel AI SDK to the workspace's canonical Stagehand code-mode MCP server | ||
| over stdio. One MCP client remains open for the complete `generateText` call, so repeated | ||
| `code_execute` calls share the same browser context. | ||
|
|
||
| The MCP child reads the normal code-mode startup environment. Set `STAGEHAND_BROWSER=local` for a | ||
| local headless browser or `STAGEHAND_BROWSER=browserbase` with Browserbase credentials for a remote | ||
| browser. | ||
|
|
||
| ```bash | ||
| pnpm --filter @browserbasehq/stagehand-integrations build | ||
| STAGEHAND_BROWSER=local pnpm --filter @browserbasehq/stagehand-integrations-example-vercel smoke | ||
| ``` | ||
|
|
||
| See [`src/agent.ts`](./src/agent.ts) for the reusable connection lifecycle and | ||
| [`src/e2e.ts`](./src/e2e.ts) for a real model-driven example. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| { | ||
| "name": "@browserbasehq/stagehand-integrations-example-vercel", | ||
| "version": "4.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "scripts": { | ||
| "e2e": "tsx src/e2e.ts", | ||
| "smoke": "tsx src/smoke.ts", | ||
| "typecheck": "tsc --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@ai-sdk/groq": "catalog:", | ||
| "@ai-sdk/mcp": "catalog:", | ||
| "@browserbasehq/stagehand-integrations": "workspace:*", | ||
| "ai": "catalog:" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/node": "catalog:", | ||
| "tsx": "catalog:", | ||
| "typescript": "catalog:" | ||
| }, | ||
| "engines": { | ||
| "node": ">=22.18.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| import { fileURLToPath } from "node:url"; | ||
| import { createMCPClient, type MCPClient } from "@ai-sdk/mcp"; | ||
| import { Experimental_StdioMCPTransport } from "@ai-sdk/mcp/mcp-stdio"; | ||
| import { STAGEHAND_CODEMODE_SKILL } from "@browserbasehq/stagehand-integrations/codemode"; | ||
| import { generateText, stepCountIs, type LanguageModel } from "ai"; | ||
|
|
||
| const DEFAULT_STDIO_SERVER_PATH = fileURLToPath( | ||
| new URL("../../../dist/codemode/stdio-server.mjs", import.meta.url), | ||
| ); | ||
|
|
||
| export type StagehandMcpBinding = { | ||
| client: MCPClient; | ||
| tools: Awaited<ReturnType<MCPClient["tools"]>>; | ||
| }; | ||
|
|
||
| export type StagehandAgentResult = { | ||
| text: string; | ||
| toolNames: string[]; | ||
| }; | ||
|
|
||
| export async function createStagehandMcpBinding( | ||
| stdioServerPath = DEFAULT_STDIO_SERVER_PATH, | ||
| ): Promise<StagehandMcpBinding> { | ||
| const client = await createMCPClient({ | ||
| transport: new Experimental_StdioMCPTransport({ | ||
| command: process.execPath, | ||
| args: [stdioServerPath], | ||
| env: definedEnvironment(), | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| }), | ||
| }); | ||
|
|
||
| return { | ||
| client, | ||
| tools: await client.tools(), | ||
| }; | ||
| } | ||
|
|
||
| export async function runStagehandAgent( | ||
| model: LanguageModel, | ||
| prompt: string, | ||
| ): Promise<StagehandAgentResult> { | ||
| const { client, tools } = await createStagehandMcpBinding(); | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| try { | ||
| const result = await generateText({ | ||
| model, | ||
| system: STAGEHAND_CODEMODE_SKILL, | ||
| prompt, | ||
| tools, | ||
| stopWhen: stepCountIs(8), | ||
| }); | ||
| return { | ||
| text: result.text, | ||
| toolNames: result.steps.flatMap((step) => step.toolCalls.map((call) => call.toolName)), | ||
| }; | ||
| } finally { | ||
| await client.close(); | ||
| } | ||
| } | ||
|
|
||
| function definedEnvironment(): Record<string, string> { | ||
| return Object.fromEntries( | ||
| Object.entries(process.env).filter( | ||
| (entry): entry is [string, string] => entry[1] !== undefined, | ||
| ), | ||
| ); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { groq } from "@ai-sdk/groq"; | ||
| import { runStagehandAgent } from "./agent.js"; | ||
|
|
||
| process.env.STAGEHAND_BROWSER ??= "local"; | ||
|
|
||
| const result = await runStagehandAgent( | ||
| groq(process.env.VERCEL_STAGEHAND_MODEL ?? "openai/gpt-oss-120b"), | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. swap this to opus 5 |
||
| [ | ||
| "Use code_execute exactly twice.", | ||
| "First navigate to https://example.com, open one additional blank tab, then restore the Example Domain page as active.", | ||
| "Second return the active page title and total context page count.", | ||
| "Report the title and count in your final answer.", | ||
| ].join(" "), | ||
| ); | ||
|
|
||
| if ( | ||
| result.toolNames.length !== 2 || | ||
| result.toolNames.some((name) => name !== "code_execute") || | ||
| !result.text.includes("Example Domain") || | ||
| !result.text.includes("2") | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| ) { | ||
| throw new Error(`Unexpected agent result: ${JSON.stringify(result)}`); | ||
| } | ||
|
|
||
| process.stdout.write(`${JSON.stringify({ status: "PASS", ...result })}\n`); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| import { createStagehandMcpBinding } from "./agent.js"; | ||
| import type { Tool } from "ai"; | ||
|
|
||
| process.env.STAGEHAND_BROWSER ??= "local"; | ||
|
|
||
| const { client, tools } = await createStagehandMcpBinding(); | ||
| try { | ||
| const names = Object.keys(tools); | ||
| if (names.length !== 1 || names[0] !== "code_execute") { | ||
| throw new Error(`Expected one code_execute tool, got ${names.join(", ")}`); | ||
| } | ||
|
|
||
| const tool = tools.code_execute as Tool<{ code: string }, unknown>; | ||
| if (typeof tool?.execute !== "function") { | ||
| throw new Error("Vercel AI SDK did not expose code_execute as executable."); | ||
| } | ||
| const description = | ||
| typeof tool.description === "function" ? tool.description({ context: {} }) : tool.description; | ||
| if (!description?.includes("Stagehand V4 code-mode syntax")) { | ||
| throw new Error("code_execute did not include the canonical Stagehand guidance."); | ||
| } | ||
|
|
||
| const first = await tool.execute( | ||
| { | ||
| code: ` | ||
| await page.goto("https://example.com", { waitUntil: "load" }); | ||
| await context.newPage(); | ||
| await context.setActivePage(page); | ||
| return { title: await page.title(), pages: (await context.pages()).length }; | ||
| `, | ||
| }, | ||
| { context: {}, messages: [], toolCallId: "vercel-smoke-1" }, | ||
| ); | ||
| const second = await tool.execute( | ||
| { | ||
| code: `return { title: await page.title(), pages: (await context.pages()).length };`, | ||
| }, | ||
| { context: {}, messages: [], toolCallId: "vercel-smoke-2" }, | ||
| ); | ||
|
|
||
| const firstText = JSON.stringify(first); | ||
| const secondText = JSON.stringify(second); | ||
| if (!firstText.includes("Example Domain") || !secondText.includes('"pages":2')) { | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| throw new Error(`Expected browser state to persist across calls: ${firstText} ${secondText}`); | ||
| } | ||
|
|
||
| process.stdout.write( | ||
| `${JSON.stringify({ status: "PASS", tools: names, statePersisted: true })}\n`, | ||
| ); | ||
| } finally { | ||
| await client.close(); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "extends": "../../../../tsconfig.json", | ||
| "compilerOptions": { | ||
| "module": "NodeNext", | ||
| "moduleResolution": "NodeNext", | ||
| "noEmit": true, | ||
| "target": "ES2022", | ||
| "types": ["node"] | ||
| }, | ||
| "include": ["src/**/*.ts"] | ||
| } |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.