fix(ci): validate the Version PR and preflight its release evidence

[FelixTJDietrich]

Problem

version-pr.yml said it outright: "GITHUB_TOKEN-authored updates do not trigger CI, so release automation uses the ruleset bypass." The Version PR carried no checks and merged through a standing bypass, so the one commit whose merge cuts a release was the one commit nothing looked at before it landed.

Pressure-testing that framing first: after #1743 the version commit is not unvalidated. cicd.yml gates Quality, Security, Test, Compose, Actionlint and Zizmor on version-bump == 'true' for pushes, which is exactly the version commit, and release.yml starts only on a successful CI/CD run. #1743's upstream scan runs on every non-pull_request event, so the pinned digests are covered there too. The remaining cost was timing — a failure lands on red main and blocks the release instead of on a red PR — plus a standing ruleset bypass with no expiry.

But subject parity is not check parity. verify-release-evidence.ts evaluates far more than the vulnerability policy, and a release was still the only thing that ever produced an evidence bundle. Three checks could fail for the first time at a release and none is structurally release-only:

• SBOM triple validation — nothing generated Syft/SPDX/CycloneDX before a release, so a Syft bump or a surprising image shape surfaces there.
• Licence report binding — nothing ran trivy --scanners license before a release.
• Vulnerability policy on linux/arm64 for the images we build — findings and exceptions are matched per platform. fix(ci): scan the pinned upstream images before the release, not at it #1743 took both platforms of the pinned upstream digests, because their remedy is a digest bump that arrives in a pull request; the images we build are still scanned on linux/amd64 only, and vulnerability-remediation.mdx on main already names this preflight as what covers the other half.

Fix

The Version PR gets real CI, with no new credential. workflow_dispatch and repository_dispatch are the two documented exceptions to the GITHUB_TOKEN no-new-run rule, so version-pr.yml starts CI/CD on changeset-release/main with the same token. scripts/dispatch-version-pr-ci.ts applies one rule — every Version PR head commit gets a run — by asking whether the head already has one, so a missed dispatch heals on the next push to main and a push that changed nothing costs nothing. The dispatched run reports CI Status Gate onto the branch head, which is the pull request's head commit, so the Version PR can carry required checks like any other.

The release gate becomes a re-verification. A new Release evidence preflight job generates a real bundle over the images its own run built and runs verify-release-evidence.ts twice — once writing the validation documents, once re-deriving them. Parity is by construction, not by a list: the same generator, the same verifier, and the verifier performs every check in every mode except the two guarded by mode === "verify-signatures".

One generator, one resolver. A second copy of the evidence-generation shell is exactly the drift this removes, so digest resolution and bundle generation move out of release.yml into scripts/resolve-release-images.ts and scripts/generate-release-evidence.ts, which release.yml and the preflight both call. Neither redefines the subject set: the first-party half comes from planSubjects, the upstream half from #1743's planUpstreamSubjects. Platform resolution reuses #1743's isImageIndex, so a release subject that is a single manifest fails rather than falling back to the index digest.

Check parity

Release evidence check	Can a release be the first to run it?
Manifest matches the inventory, both platforms, canonical order	No — check-release-image-inventory.ts on every PR, plus a test feeding the real generator's manifest to the real verifier
Syft/SPDX/CycloneDX describe the same subject and packages	No — preflight
Trivy licence report bound to its subject	No — preflight
Vulnerability policy, linux/amd64	No — build gate, upstream scan, preflight
Vulnerability policy, linux/arm64	No — ci-security-scan.yml for the pinned upstream digests, preflight for the images we build
Platform digest is a member of its published index	No — preflight
Validation documents re-derive unchanged	No — preflight verifies twice
SBOM attestation matches the durable evidence	Yes, structurally — cosign attest creates it during the release
Index signature and build provenance carry the release identity	Yes, structurally — the Fulcio identity is release.yml@refs/heads/main
The manifest names a release version	Yes, structurally — only plan-release.ts decides the tag

ci-contract.test.ts asserts the two signature entries are the only mode-conditional behaviour in the verifier, so a new check gated on anything else fails the contract test rather than a release. docs/contributor/release-management.mdx § Nothing may fail for the first time at a release is the durable home for the reasoning.

The upstream images are judged twice on the Version PR, deliberately

The upstream scan is a minute of Trivy answering the one question a digest bump can break; the preflight is the whole evidence gate answering it while verifying a bundle. They cannot disagree — one planner, one evaluator, one policy file, and the same selectPlatformDigest over the same index — so the cost is a duplicated scan, not a second opinion. Dropping the fast one takes the quick answer away from the PR that most needs it; dropping the upstream half of the bundle leaves a bundle validateManifest rejects. Written down in release-management.mdx so it reads as a choice rather than an oversight.

Failure modes reasoned about, since release.yml cannot be run

• Manifest shape. Key order, subject order and generatedAt format are preserved deliberately; durationSeconds stays a number. Asserted by the generator test round-tripping through validateManifest.
• Resolution. The retry budget (24 × 5 s) is unchanged, and an image the registry will not answer for still fails the run rather than being dropped — an incomplete subject set is what the gate exists to catch.
• Fail-closed platform resolution. A single manifest and an index missing a platform are separated, both throw, and the preflight surfaces them: the script exits non-zero under set -euo pipefail, nothing catches it.
• Step outputs. resolve-release-images.ts writes <image>-digest= for every image; release.yml still reads the three it needs, and subjects.sha256 reads the widened TSV.
• Stale dispatch inputs. gh workflow run validates inputs against the workflow file on the target ref. changeset-release/main does not exist today and is rebuilt from main's tip whenever changesets runs, so it can never predate the release-preflight input.
• Concurrency. The group now includes the event name, so a dispatch can never cancel a push on the same ref — a push produces the images a release promotes. cancel-in-progress is on for pull requests (unchanged) and dispatches; merge groups and pushes stay uncancellable.
• Rate limits. The preflight pulls the four upstream images from Docker Hub, as the release does. If anonymous limits ever bite, they bite the preflight first — the safe place.
• Cost. The preflight is a Syft and Trivy pass over eight images on two architectures, so it runs only where its answer changes a decision: the Version PR, and any manual dispatch that ticks release-preflight.

Not included

No changeset: verify-changesets scopes SHIPPED_PATHS to server, webapp and docker, and this touches only .github/, scripts/ and docs/. No vulnerability-policy exceptions — #1743 owns those. No gate was weakened. Nothing was added to the release-images path filter's transitive import closure, so #1743's importClosure assertion still holds unchanged.

Maintainer action

Nothing is required for this to work. Once it has run once, the standing ruleset bypass on the Version PR has no remaining purpose and can be retired, in this order:

1. Merge this, let one Version PR appear and confirm CI Status Gate lands green on its head commit.
2. Confirm nothing else consumes that bypass — I cannot read the ruleset.
3. Remove the bypass entry, keeping CI Status Gate, Actionlint and Zizmor required.

Doing 3 before 1 blocks releases behind checks that are not yet reporting.

Verification

pnpm run format and pnpm run check both pass. ci-contract.test.ts passes as a whole — 27 tests, including #1743's importClosure and both-platforms assertions. New node:test coverage: the resolver's retry, digest validation and TSV hand-off; the generator's real manifest accepted by the real validateManifest; the dispatcher's branch derivation and run decisions.

---

Model: Claude Fable 5. Harness: Claude Code.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 20 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 42a3939a-86e5-42b3-8353-93ab248dd1a1

📥 Commits

Reviewing files that changed from the base of the PR and between aa0ebaf and a6f1cd7.

📒 Files selected for processing (14)

• .github/workflows/cicd.yml
• .github/workflows/release.yml
• .github/workflows/version-pr.yml
• docs/contributor/ci-cd.mdx
• docs/contributor/release-management.mdx
• docs/contributor/vulnerability-remediation.mdx
• scripts/ci-contract.test.ts
• scripts/dispatch-version-pr-ci.test.ts
• scripts/dispatch-version-pr-ci.ts
• scripts/generate-release-evidence.test.ts
• scripts/generate-release-evidence.ts
• scripts/resolve-release-images.test.ts
• scripts/resolve-release-images.ts
• scripts/verify-release-evidence.ts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Approved automatically: @FelixTJDietrich is listed in the REVIEW_POLICY_MAINTAINERS repository variable, which the repository treats as satisfying the review requirement. See the review policy in docs/contributor/ci-cd.mdx.

[github-actions]

📚 Documentation Preview

Preview has been removed (PR closed)

[github-actions]

🧩 Storybook Preview

Preview has been removed (PR closed)

The base branch was changed.

[github-actions]

Approved automatically: @FelixTJDietrich is listed in the REVIEW_POLICY_MAINTAINERS repository variable, which the repository treats as satisfying the review requirement. See the review policy in docs/contributor/ci-cd.mdx.

[github-actions]

Approved automatically: @FelixTJDietrich is listed in the REVIEW_POLICY_MAINTAINERS repository variable, which the repository treats as satisfying the review requirement. See the review policy in docs/contributor/ci-cd.mdx.