feat(vscode): add Playwright-based e2e test infrastructure by MaxKless · Pull Request #3109 · nrwl/nx-console

MaxKless · 2026-04-08T09:04:57Z

Summary

Adds apps/vscode-e2e project with Playwright + Electron for e2e testing the VS Code extension against a real VS Code instance and real Nx workspace
Architecture ported from GitLens: HTTP server runner injected into VS Code extension host, file-based evaluator for vscode API access, worker-scoped fixtures with isolated workspaces
Includes page objects for activity bar, sidebar, quick pick, and Nx Console-specific tree view interactions
Smoke test covers: extension activation, command registration, activity bar visibility, project tree loading, and project target expansion

Key design decisions

Playwright _electron.launch() drives VS Code directly (no Selenium/WebDriver)
Worker-scoped fixtures — one VS Code instance + Nx workspace per Playwright worker
File-based evaluator — runner writes server URL to temp file, test side polls for it (extension host stderr isn't accessible from Playwright's Electron process)
Real Nx workspaces via create-nx-workspace (reuses libs/shared/e2e-utils)
Git must remain enabled in VS Code settings — Nx Console depends on the Git extension API

New dependencies

@playwright/test 1.52.0
playwright 1.52.0

Test plan

npx playwright test -c apps/vscode-e2e/playwright.config.ts passes locally
Verify CI doesn't regress (vscode-e2e not in affected targets yet — manual trigger only)
Future: add .github/workflows/vscode-e2e.yml with workflow_dispatch for on-demand CI runs

🤖 Generated with Claude Code

Adds a new `vscode-e2e` project with Playwright + Electron support for end-to-end testing the VS Code extension against a real VS Code instance with a real Nx workspace. Architecture (ported from GitLens): - Runner HTTP server injected into VS Code extension host via --extensionTestsPath, exposes POST /invoke for vscode API access - File-based evaluator discovers the runner's URL via temp marker files - Worker-scoped Playwright fixture creates an isolated Nx workspace, launches VS Code, connects the evaluator, and provides page objects - Page objects for activity bar, sidebar, quick pick, and Nx Console specific tree view interactions Smoke test covers: extension activation, command registration, activity bar visibility, project tree loading, and project target expansion. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

nx-cloud-snapshot · 2026-04-08T09:05:43Z

View your CI Pipeline Execution ↗ for commit 42d7c7e

Command	Status	Duration	Result
`nx affected --targets=lint,test,build,e2e-ci,ty...`	✅ Succeeded	1m 20s	View ↗
`nx-cloud record -- node tools/scripts/check-pin...`	✅ Succeeded	<1s	View ↗
`nx-cloud record -- yarn nx sync:check`	✅ Succeeded	8s	View ↗
`nx-cloud record -- yarn nx run-many -t ktfmtFormat`	✅ Succeeded	5m 35s	View ↗
`nx run-many -t ktfmtFormat`	✅ Succeeded	5m 31s	View ↗
`nx-cloud record -- yarn nx format:check --verbose`	✅ Succeeded	5s	View ↗

☁️ Nx Cloud last updated this comment at 2026-04-10 11:40:16 UTC

apps/vscode-e2e/runner/src/index.ts

+        }
+
+        res.writeHead(200, { 'Content-Type': 'application/json' });
+        res.end(JSON.stringify(response));


graphite-app · 2026-04-08T09:08:28Z

apps/vscode-e2e/page-objects/components/quick-pick.ts

+  }
+
+  async execute(command: string): Promise<void> {
+    await this.page.keyboard.press('Meta+Shift+P');


Platform-specific keyboard shortcut breaks on Windows/Linux

Meta+Shift+P only works on macOS. On Windows and Linux, the command palette shortcut is Control+Shift+P. This will fail on non-macOS platforms.

Fix: Use cross-platform modifier or detect platform:

const modifier = process.platform === 'darwin' ? 'Meta' : 'Control'; await this.page.keyboard.press(`${modifier}+Shift+P`);

Suggested change

await this.page.keyboard.press('Meta+Shift+P');

const modifier = process.platform === 'darwin' ? 'Meta' : 'Control';

await this.page.keyboard.press(`${modifier}+Shift+P`);

Spotted by Graphite

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2026-04-08T09:48:18Z

apps/vscode-e2e/specs/smoke.test.ts

+  await expect(projectsSection.locator('.monaco-list-row')).not.toHaveCount(2, {
+    timeout: 10_000,
+  });


Test logic error: This assertion waits for the row count to NOT be 2, but this doesn't guarantee expansion has occurred. If the initial count is 1 (just the collapsed project), the assertion passes immediately since 1 ≠ 2, even though expansion hasn't happened yet. This can cause race conditions where line 40's assertion expect(rowCount).toBeGreaterThan(2) fails.

// Wait for expansion to complete by checking for minimum expected rows await expect(projectsSection.locator('.monaco-list-row')).toHaveCount( 3, { timeout: 10_000 } ).catch(() => {}); // or use a more specific count if known // Or wait for a specific target to appear await expect( projectsSection.locator('.monaco-list-row').nth(1) ).toBeVisible({ timeout: 10_000 });

Suggested change

await expect(projectsSection.locator('.monaco-list-row')).not.toHaveCount(2, {

timeout: 10_000,

});

await expect(projectsSection.locator('.monaco-list-row')).toHaveCount(

3,

{ timeout: 10_000 }

).catch(() => {});

Spotted by Graphite

Is this helpful? React 👍 or 👎 to let us know.

.github/workflows/vscode-e2e.yml

graphite-app · 2026-04-08T09:55:08Z

apps/vscode-e2e/runner/src/index.ts

+      }
+
+      let body = '';
+      req.on('data', (chunk: string) => {


Type annotation is incorrect. The data event emits Buffer objects, not strings. This will cause incorrect concatenation behavior.

req.on('data', (chunk: Buffer) => { body += chunk.toString(); });

Or remove the type annotation and let TypeScript infer it.

Suggested change

req.on('data', (chunk: string) => {

req.on('data', (chunk: Buffer) => {

Spotted by Graphite

Is this helpful? React 👍 or 👎 to let us know.

nx-cloud-snapshot

Important

At least one additional CI pipeline execution has run since the conclusion below was written and it may no longer be applicable.

Nx Cloud is proposing a fix for your failed CI:

We replaced --experimental-strip-types with --loader ts-node/esm in the vscode-e2e:test command to fix the failure on Node.js v20, where that flag does not exist (it was introduced in v22.6.0). We also updated the spec's import specifier from ./vscode-e2e-runtime.ts to ./vscode-e2e-runtime.js, which is the standard TypeScript ESM convention that ts-node/esm uses to resolve the corresponding .ts source file at runtime.

Warning

❌ We could not verify this fix.

diff --git a/apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts b/apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts
index 96c31b86..a5f24668 100644
--- a/apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts
+++ b/apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts
@@ -5,7 +5,7 @@ import {
   getMarkerFilePath,
   getMarkerId,
   getWorkerDisplay,
-} from './vscode-e2e-runtime.ts';
+} from './vscode-e2e-runtime.js';
 
 test('marker ids and file paths are worker-specific', () => {
   const firstWorkerId = getMarkerId(0);
diff --git a/apps/vscode-e2e/project.json b/apps/vscode-e2e/project.json
index f348d25b..5c72c26b 100644
--- a/apps/vscode-e2e/project.json
+++ b/apps/vscode-e2e/project.json
@@ -29,7 +29,7 @@
       ]
     },
     "test": {
-      "command": "node --test --experimental-strip-types apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts"
+      "command": "node --loader ts-node/esm --test apps/vscode-e2e/fixtures/vscode-e2e-runtime.spec.ts"
     },
     "build-runner": {
       "command": "esbuild apps/vscode-e2e/runner/src/index.ts --bundle --outfile=dist/apps/vscode-e2e/runner/index.js --platform=node --external:vscode --format=cjs",

Or Apply changes locally with:

npx nx-cloud apply-locally qZ4m-X5LE

Apply fix locally with your editor ↗ View interactive diff ↗

_{🎓 Learn more about Self-Healing CI on nx.dev}

.github/workflows/vscode-e2e.yml

…ntain permissions' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

leosvelperez · 2026-04-10T11:27:18Z

.github/workflows/vscode-e2e.yml

+
+      - name: Upload VS Code E2E videos
+        if: always()
+        uses: actions/upload-artifact@v4.6.2


Should we pin the commit hash here as well?

github-advanced-security bot found potential problems Apr 8, 2026

View reviewed changes

apps/vscode-e2e/runner/src/index.ts Dismissed Show dismissed Hide dismissed

apps/vscode-e2e/runner/src/index.ts

}

res.writeHead(200, { 'Content-Type': 'application/json' });

res.end(JSON.stringify(response));

graphite-app bot reviewed Apr 8, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

fix(vscode): stabilize playwright e2e harness

bec9df2

graphite-app bot reviewed Apr 8, 2026

View reviewed changes

ci(vscode): add manual e2e workflow

7363d1d

github-advanced-security bot found potential problems Apr 8, 2026

View reviewed changes

.github/workflows/vscode-e2e.yml Fixed Show fixed Hide fixed

graphite-app bot reviewed Apr 8, 2026

View reviewed changes

nx-cloud-snapshot bot reviewed Apr 8, 2026

View reviewed changes

MaxKless added 3 commits April 10, 2026 10:37

fix(vscode-e2e): run runtime test on node 20

cf89bc4

fix(vscode-e2e): address review feedback

df23730

feat(vscode-e2e): upload recorded e2e videos

88ab469

github-advanced-security bot found potential problems Apr 10, 2026

View reviewed changes

.github/workflows/vscode-e2e.yml Fixed Show fixed Hide fixed

Potential fix for pull request finding 'CodeQL / Workflow does not co…

a04ba90

…ntain permissions' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

leosvelperez approved these changes Apr 10, 2026

View reviewed changes

ci(vscode-e2e): pin upload-artifact action

42d7c7e

MaxKless merged commit e47823a into master Apr 10, 2026
7 checks passed

MaxKless deleted the trusting-cause branch April 10, 2026 12:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(vscode): add Playwright-based e2e test infrastructure#3109

feat(vscode): add Playwright-based e2e test infrastructure#3109
MaxKless merged 8 commits intomasterfrom
trusting-cause

MaxKless commented Apr 8, 2026

Uh oh!

nx-cloud-snapshot bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

graphite-app bot Apr 8, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

graphite-app bot Apr 8, 2026

Uh oh!

Uh oh!

graphite-app bot Apr 8, 2026

Uh oh!

nx-cloud-snapshot bot left a comment •

edited

Loading

Uh oh!

Uh oh!

leosvelperez Apr 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	await this.page.keyboard.press('Meta+Shift+P');
	const modifier = process.platform === 'darwin' ? 'Meta' : 'Control';
	await this.page.keyboard.press(`${modifier}+Shift+P`);

-  await expect(projectsSection.locator('.monaco-list-row')).not.toHaveCount(2, {
-    timeout: 10_000,
-  });
+  await expect(projectsSection.locator('.monaco-list-row')).toHaveCount(
+,
+    { timeout: 10_000 }
+  ).catch(() => {});

	req.on('data', (chunk: string) => {
	req.on('data', (chunk: Buffer) => {

Conversation

MaxKless commented Apr 8, 2026

Summary

Key design decisions

New dependencies

Test plan

Uh oh!

nx-cloud-snapshot bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

graphite-app bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

graphite-app bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

graphite-app bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

nx-cloud-snapshot bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Nx Cloud is proposing a fix for your failed CI:

Uh oh!

Uh oh!

leosvelperez Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nx-cloud-snapshot bot commented Apr 8, 2026 •

edited

Loading

nx-cloud-snapshot bot left a comment •

edited

Loading