Skip to content

test(playwright): Add harness contract baseline - #37913

Draft
shortstacked wants to merge 2 commits into
masterfrom
devp-1064-add-test-harness-contract-tests-and-a-baseline
Draft

test(playwright): Add harness contract baseline#37913
shortstacked wants to merge 2 commits into
masterfrom
devp-1064-add-test-harness-contract-tests-and-a-baseline

Conversation

@shortstacked

@shortstacked shortstacked commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add six fixture consumer contracts for API-only, UI-only, combined, service-only, body-failure, and bootstrap-failure cases.
  • Run the consumers in a real Playwright subprocess so the tests verify process exit codes, reports, browser cleanup, and support-server cleanup.
  • Add an isolated Vitest configuration for the browser-dependent harness tests.
  • Record the current fixture dependency graph, test inventory, requirements migration scope, and evidence limits.
  • Record current unwanted behavior without making it a required contract. API-only and service-only consumers currently start a browser. Combined setup currently performs duplicate resets.

How to test

Run from packages/testing/playwright:

pnpm test:harness --reporter=verbose --silent=false
pnpm test:unit
pnpm typecheck
pnpm lint

The harness suite requires Chromium. Install it with pnpm install-browsers if it is not available.

The harness suite uses loopback support servers. It does not start an n8n application or a Docker stack. Real Docker and application sentinels remain separate follow-up work.

Expected results:

  • Harness contracts: 6 passed.
  • Existing Playwright unit tests: 82 passed.
  • Typecheck and lint pass.

Related Linear tickets, Github issues, and Community forum posts

Review / Merge checklist

  • I have seen this code, I have run this code, and I take responsibility for this code.
  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with Backport to Beta, Backport to Stable, or Backport to v1 (if the PR is an urgent fix that needs to be backported)

Non-negotiable constraints

This baseline must not reduce selected tests, assertions, workloads, or isolation to save CI time. It must not add unexplained skips, retries, timeout increases, or silent fallbacks. It must keep setup failures and missing evidence visible.

PR Summary generated by AI

This PR adds a fast, isolated baseline for the Playwright fixture refactor. It verifies fixture wiring and lifecycle cleanup without claiming to verify Docker, n8n readiness, or deployment compatibility.

Review in cubic

@n8n-assistant

n8n-assistant Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

PR review overview

Based on ownership of the 8 changed files in this PR:

Ownership Files owned Share Source code Test files Misc
@n8n-io/qa-dx 7 88% +0 / -0 +644 / -0 +0 / -0
@n8n-io/catalysts 1 13% +0 / -0 +0 / -0 +3 / -0
Total 8 100% +0 / -0 +644 / -0 +3 / -0

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files

Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic

@n8n-assistant n8n-assistant Bot added the n8n team Authored by the n8n team label Sep 5, 2026
@codecov

codecov Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds a Playwright harness contract baseline: six synthetic consumer tests, isolated Vitest config, support servers, and baseline documentation. Test-only change, no production behavior; bounded and clearly beneficial.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed n8n team Authored by the n8n team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant