Run automated conformance suite in CI

[russlan23]

Describe your change

Adds a Playwright contribution check for the FDC3 conformance framework. The check starts the conformance app and the FDC3 for Web reference Desktop Agent, launches the runner inside the Desktop Agent, selects the complete non-manual pack, and fails with the underlying conformance failure messages when any test fails.

The browser runner now exposes machine-readable status and totals after the Mocha run. While exercising the pack, the check also exposed three setup and cleanup problems in the recently added destructured-method tests; this change supplies the required getAppMetadata argument, waits for freshly opened mock apps to initialize, awaits the close broadcast, and uses the existing mock-processing timeout.

Manual intent-resolver and channel-selector tests remain outside this automated check.

Validation performed:

• Complete non-manual conformance pack against the reference Desktop Agent: 115 passed, 0 failed
• Isolated fdc3.destructuredMethods pack: 8 passed, 0 failed
• npm run test
• npm run lint (passes with existing workbench warnings)
• npm run build --workspace fdc3-conformance
• Conformance-package ESLint and Prettier checks
• npm run test:ci --workspace fdc3-conformance -- --list

Related Issue

Resolves #2045.

Contributor License Agreement

• I acknowledge that a contributor license agreement is required and that I have one in place or will seek to put one in place ASAP.

Review Checklist

• Issue: The contribution is linked to Make FDC3 conformance test framework run under CI as a contribution test #2045.
• CHANGELOG: A CHANGELOG.md entry is included.
• API changes: No FDC3 API is changed.
• Context types: No Context type is added or modified.
• Intents: No Intent is added or modified.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: russlan23 / name: russlan23 (94990af)

[netlify]

✅ Deploy Preview for fdc3 ready!

Name	Link
🔨 Latest commit	d540915
🔍 Latest deploy log	https://app.netlify.com/projects/fdc3/deploys/6a906b2fda9b150008db77bc
😎 Deploy Preview	https://deploy-preview-2048.preview-fdc3.finos.org
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[russlan23]

/easycla

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.11%. Comparing base (83d839d) to head (b633572).
⚠️ Report is 56 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2048   +/-   ##
=======================================
  Coverage   95.11%   95.11%           
=======================================
  Files          73       73           
  Lines        4196     4196           
  Branches      581      670   +89     
=======================================
  Hits         3991     3991           
  Misses        205      205           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Roaders]

This looks promising as an initial way to get these tests running as part of the CI process but we must be sure that the build will fail if the tests fail. As the tests timeout at the moment but this build is still fine I am not sure this is the case.

[russlan23]

Addressed in 335fc22. I added an explicit result gate that throws unless the conformance run reports passed, has zero failures, executes at least one test, and completes every reported test. I also added negative coverage for failed or timed-out and incomplete runs.

Local verification: npm run test:ci --workspace fdc3-conformance — 4 passed in 5.9m. A conformance failure, incomplete run, or the existing outer Playwright timeout now returns a non-zero exit from the command used directly by the CI job.

[russlan23]

Addressed in 335fc22. I added an explicit result gate that throws unless the conformance run reports , has zero failures, executes at least one test, and completes every reported test. I also added negative coverage for failed/timed-out and incomplete runs.\n\nLocal verification:

fdc3-conformance@3.0.0-alpha.2 test:ci
playwright test

Running 4 tests using 1 worker

✓ 1 e2e/conformance.spec.ts:9:1 › runs all automated conformance tests against the reference Desktop Agent (5.8m)
✓ 2 e2e/conformanceResults.spec.ts:10:3 › conformance result gate › accepts a completed passing run (0ms)
✓ 3 e2e/conformanceResults.spec.ts:22:3 › conformance result gate › rejects a failed or timed-out conformance test (1ms)
✓ 4 e2e/conformanceResults.spec.ts:34:3 › conformance result gate › rejects a run that ends without completing every test (0ms)

Slow test file: e2e/conformance.spec.ts (5.8m)
Consider running tests from slow files in parallel. See: https://playwright.dev/docs/test-parallel
4 passed (5.8m) — 4 passed in 5.9m. A conformance failure, incomplete run, or the existing outer Playwright timeout now returns a non-zero exit from the command used directly by the CI job.

[russlan23]

Rebased this branch onto current main after #2143 merged, so the overlapping destructured-method setup and cleanup fix now drops out of this PR. Mocha 11 also requires its published browser bundle here; importing the Node entry caused Vite to externalize node:util and left the automated test selector empty. That compatibility fix is in d540915, and the conformance package build passes.

The full gate now executes and reports three failures on current main (cb00631d) after #1942:

• RaiseIntentExistingInstanceRequired: two instances remain after newInstance=false, expected one.
• RaiseIntentFailExistingInstanceRequired: the call does not surface TargetInstanceUnavailable as expected.
• IntentContextMetadataWithAppMetadata: the delivered traceId is regenerated instead of preserving intent-trace-456.

The three result-gate unit tests still pass. I have not mixed changes to #1942 behavior into this CI PR; I will keep those fixes separate if maintainers want a follow-up.