Rebuild swift-ci.yml from the ground up and reduce the repository to what CI needs - #551
Merged
Conversation
TX-5 of the reduction census (#515). `Tools/institute-ci` and `Tools/repository-policy` are gone; what they implemented now lives in the packages that own it, and this repository consumes those packages the same way every other consumer does. **The binary channel.** `actions/institute-ci` and swift-ci.yml's two inline bootstrap blocks provision swift-institute/institute-continuous-integration at a pinned commit instead of a directory in this checkout. The cache key was the git tree OID of `Tools/institute-ci`; it is now the pinned commit, which is the same kind of fact — exact content identity — and needs no working tree to compute. Provenance verification is unchanged and still fail-closed. **The policy data stays.** `Tools/repository-policy/Policy/` moves to `policy/` at the root, beside `canon/`: it is fleet policy this repository authors, not engine source. The engine now builds where it is checked out, so the copy-the-engine-in-beside-the-data step is gone, and `--surface-policy` is passed explicitly rather than falling through to a package-relative default that only resolved because of that co-location. **The shipped-bytes controls survive the move.** The suites that extract swift-ci.yml's aggregator, subject resolver, linter installer, and the system-dependency action's step travel with the reader they use, and lint-validator-fixtures.yml names this checkout to them through INSTITUTE_CONTROL_PLANE_ROOT. Without it they decline rather than report green against a root they guessed. **tools-tests.yml is retired.** It ran the two packages' suites through `workspace package test` on a repository that no longer contains them; each package now runs its own. No required status check named it. The `sources-revision` pins here are the pre-port head of institute-continuous-integration and MUST advance to the port's merge SHA before this lands, or the provisioned binary will not carry the faces these workflows call.
An audit of the previous commit found five defects it introduced. All are fixed here. **ci-ok built from one directory and copied from another.** The cold bootstrap built `.institute-ci-sources` but copied the binary out of `institute-receipt/.build` — the control-plane checkout, which carries no Swift package at all any more. The `Tools/institute-ci` segment had been stripped from the path instead of the whole prefix being replaced. Every cold ci-ok run would have failed at `cp`. **The pin had seven copies and only prose binding them.** It is now `.institute-ci-pin` at the repository root, read by the composite action (relative to its own path), by both swift-ci.yml bootstrap blocks, by lint-validator-fixtures.yml, and by swift-docs.yml. Before this the cache key `institute-ci-<pin>-…` could split silently on a partial advance: the plan job minting a manifest at one revision while ci-ok verified at another, each believing its provenance check passed. The tree OID it replaced was a single runtime-computed authority; duplicating literals gave that up, and this restores it. **One fact carried two names.** The identity step emitted `sources-tree` while carrying a commit SHA and passing it to `--sources-revision`; the composite action already called it `sources-revision`. The two channels that must agree on a cache key now spell it the same way. **Both source checkouts ran unconditionally** while their only consumer is cache-gated — a full clone added to every warm run of the universal workflow, fleet-wide, for nothing. They now carry the same cache-miss gate the composite action always had. The control-plane checkout in ci-ok stays, because it is where that job reads the pin: routing it through `needs.plan.outputs` would leave it empty exactly when plan failed and ci-ok still has to report the plan-failure finding. **A guard was credited to the wrong mechanism.** `ref:` comments claimed the branch-pin guard rejects a non-SHA pin; it reads root `Package*.swift` manifests and has never read a workflow ref. The refusal is the inline 40-hex check, and the comments now say so — a maintainer trimming duplication could have deleted it believing something else covered it. Also swept the stale Tools/ narration out of six workflows, replaced the `reviewOnly` worked example that named a class of PR this repository can no longer produce, and restored the precise test-target coordinate in the schema-correspondence README.
`.swiftlint.yml` is a live fleet dependency, not repository furniture: the universal's lint job checks this repository out at its own revision and rewrites every consumer's `parent_config` to point at this exact copy, so all 471 callers resolve their rule set from here. The rough cleanup removed it, which would have failed that job fleet-wide at "pinned Tier 1 .swiftlint.yml did not check out". `.swift-format` is narrower and worth not confusing with it. The format job reads each consumer's own file, discovered from the consumer's checkout, so this one governs this repository's sources and stands as the canonical copy the layer packages take. Restored on both counts. The pin moves to the commit where the binary first implements the verbs this workflow calls. Until #22 the extracted repository served three gitignore verbs and none of `plan`, `aggregate`, `bootstrap-manifest` or `bootstrap-verify`.
Forty duplicated step blocks across fourteen legs collapse into one checkout-and-prove action. The proof each leg was carrying — that the checkout is the revision the plan resolved — is the part worth having in one place: fourteen copies is fourteen chances to drift and one chance to be quietly dropped. central-style displaces every package-local .swift-format and .swiftlint.yml with the single central pair, at any depth. SwiftLint prefers the nearest config to each file, so a leftover nested copy would keep governing its own subtree — which is the non-uniformity this ends.
…rd it 3018 lines to 707, seventeen inputs to five, four secrets to one. The file was 40% comment by line, and most of that was history: measurements that justified a decision years of runs ago, arguments with positions nobody holds any more, and the reasoning behind branches this commit deletes. That belongs in Internal, where it already is. What stays is the traps — the ones a future editor would otherwise walk back into. The three dispatch inputs go. ci-dispatch and ci-sweep are retired, and zero of the 473 live callers passed target-repo, job or ref; between them they carried a subject-resolution ladder and a selector disjunct in all eighteen job guards. swift-version, macos-runner and enable-private-repos go too, on the same evidence: nobody passes them, so their defaults were the contract. Four secrets become one. 465 of 471 callers pass none at all, and the credential action no-ops on empty input, so the PAT fallback and the duplicate app-id spelling were configuring nothing anywhere. The six that do pass one pass exactly the private key, which is the ruled terminal profile the fleet had already converged on. Docs, the six advisory linter jobs and every tag-ref branch are gone — the first two by ruling, the third because the Institute develops main-only and those branches could never fire. Format and lint now displace any package-local configuration with the central pair before measuring. Enforcement is uniform from this commit; deleting the per-package copies afterwards is cleanup, not a cutover, and cannot redden the fleet on its own. Composite actions must carry full coordinates, not `./`: a relative action path resolves against the workspace, which in a reusable holds the consumer's package rather than this repository.
`--static-swift-stdlib` links Foundation statically, which pulls in lib_CFURLSessionInterface.a and every libcurl symbol with it. The bootstrap provisioned only uuid-dev, so the link failed with 40-odd undefined references to curl_* and took the plan job down with it — before the planner could classify anything. Nothing in the binary opens a socket. The event diff is read by shelling out to `gh`. The symbols still have to resolve for a static link, which is why this is a provisioning defect rather than a dependency one. Named in --provisioning as well as installed: the provisioning set is part of the cache entry's identity, so omitting it there would let an entry built without libcurl satisfy a key that claims to have it.
The action installed a Swift toolchain, checked out two repositories, ran a SwiftPM build through the Workspace coordinate, and provisioned apt packages — all on the critical path of the plan job, which is the first job of every run of every package in the fleet. A cache entry made that invisible until an exact key finally missed and the cold path was found not to link. Consume the published rolling release instead. Every run checks SHA256SUMS and then refuses unless the manifest's recorded source revision equals the fleet pin, so the tag is verified rather than trusted, and a pin advance whose publish has not landed stops here instead of running an older binary under a newer pin's name.
The consumer verifies the published manifest's recorded source revision against this file and refuses on disagreement, so the pin now names the revision the binary was actually built from rather than the last revision anyone built from source.
…te CI pin The action reads .institute-ci-pin relative to GITHUB_ACTION_PATH, so it sees the pin as of its own pinned revision, not the workflow's. Advancing the pin therefore has no effect until the action reference advances with it — the previous references predated the advance and refused, correctly, against the older pin.
The package-content predicate reads the event's changed files through `gh`, which authenticates from the environment only; the step never set GH_TOKEN, so the predicate refused on every pull request. Grant pull-requests: read alongside it. The 60 minute timeout was sized for the cold source build that no longer exists. Ten is generous for a download and a classification.
A called workflow cannot request more than the caller grants, and the generated caller grants actions: read and contents: read. Asking for pull-requests: read failed the run at startup, before any job was scheduled. The predicate needs no further grant: the visibility gate means every repository reaching this step is public.
The plan step's summary ended with `[ -n "$D" ] && echo …`, which is the last command in its group. On the ordinary case — nothing descheduled — the test returns 1, the group returns 1, and set -e failed the step after every output had already been written: a clean log above a bare exit 1. An action manifest's description field is itself a template, so central-style's spelling of the job-context expression in prose made the whole action fail to load with "Unrecognized named-value: 'job'". Both manifests now scan clean for templated descriptions.
swift-linter's publisher checksums MANIFEST.txt alongside the two binaries, so `sha256sum --check` fails on a missing entry rather than skipping it. The step downloaded only the binaries.
The advisory summary carried the same shape as the plan step: a test as the last command of a group that is itself the script's last command, so the ordinary empty case set the step's exit status to 1. Fixing one instance without sweeping its class left the twin in place.
The sidebar lists jobs in definition order, so grouping needs both a common prefix and the definitions themselves in that order. Legs are now Linux, then Apple, then Windows, then the quality gates, and the prefix survives the graph view's truncation — the distinguishing word comes first instead of after 'Ubuntu (Swift'. Four names still interpolated inputs.swift-version, an input the reduction deleted, so they rendered as 'Ubuntu (Swift , release)'. The version is owned by env and belongs in one place; a name that restates it is a second declaration that can drift, which is exactly what happened. Dropped rather than repaired. ci-ok keeps its name: 'ci / matrix / ci-ok' is the required status check named by the rulesets, and renaming it would silently un-gate the fleet. Every job body is byte-identical; only names and order changed.
A skipped matrix job never expands, so a name referencing matrix.platform has nothing to substitute and GitHub renders the template literally. The leg is exhaustive-tier only and skipped in nearly every run, so that unexpanded state is the one actually read.
install-system-deps, install-swift-sdk and configure-private-repos were removed in the cleanup while swift-ci.yml and ci-subject kept using them, pinned to commits on main. That resolves — a commit-pinned uses: is immortal — but it left the workflow depending on files absent from the repository's own tree, readable only by digging through history, and it split the action pins across three unrelated revisions.
The workflow referenced its own repository's actions at three unrelated commits, so 'the actions this workflow uses' was not a single reviewable state. All 25 sites now name one revision.
The reduction retires Tools/ and .github/scripts/, and 64 of the 78 deleted workflows go with them because their implementation was there. This one does not: TX-1 of the reduction census already moved the transaction model into swift-institute/institute-pull-request-transaction, and the workflow consumes it from a pinned checkout at 5528f99b. It reaches nothing this branch removes. It is also the sole distinct-reviewer route for a coenttb-authored pull request, so deleting it would have removed the means of approving both this change and every change after it.
Member
Author
|
Forward-only activation slice published:
Activation order is #10 and #5, then Institute CI #24, then this PR. No historical Python, fixture tree, support YAML, or workflow caller was restored. |
This was referenced Aug 13, 2026
# Conflicts: # .github/workflows/swift-ci.yml # .institute-ci-pin
![swift-institute-bot[bot]](https://avatars.githubusercontent.com/in/3543256?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds the universal reusable rather than continuing to refactor it, and reduces this repository to the files that serve it.
Why a rewrite
The step-by-step refactor kept failing for two reasons.
swift-ci.ymlhad an undeclared dependency closure on its own repository's tree — thirteen dangling references — and theTools/extraction had moved the package name without the capability, leaving five CLI verbs the workflow called but nothing implemented. Neither is reachable by small steps.Shape
swift-ci.yml: 3018 → ~710 lines, 17 inputs → 5, 4 secrets → 1, 16 jobs. Deleted: the three dispatch inputs,swift-version/macos-runner/enable-private-repos, thedocsjob and its five inputs, six advisory linter jobs, and every tag-ref branch. Repository overall: 865 files, −80,646 lines.Two new composite actions carry what was duplicated.
ci-subjectreplaces forty copies of the checkout + HEAD-proof + credential block across fourteen legs — fourteen places for a nine-line proof to drift, and one place for it to be silently dropped.central-stylematerialises the single.swift-formatand.swiftlint.ymlthe fleet is measured against, displacing package-local copies at any depth.Scheduling stays owned by
institute-continuous-integration:planemits the leg list and every job binds it, which is what letsci-okfail a gating leg that skipped without the plan having excluded it.The binary channel
The
planjob used to cold-buildinstitute-continuous-integrationthrough the Workspace coordinate — a ~30 minute SwiftPM build, a toolchain install, two checkouts and an apt provisioning set, on the critical path of the first job of every run of every package in the fleet. It appeared fast only while oneactions/cacheentry survived. Cache keys are exact by policy, so the first change to the sources or the provisioning set forced the cold path, and by then that path no longer linked.It is now downloaded from a published rolling release, checksum-verified, and refused unless the manifest's recorded source revision equals
.institute-ci-pin. ~30 minutes → 1.4 seconds, and the identity is stronger than a cache key rather than weaker. Producer: swift-institute/institute-continuous-integration#23.Evidence
Four probes against this branch at full tier:
swift-formatSwiftLintThe two failures are genuine package content meeting the central rule set for the first time, not CI defects —
ci-okrefuses withselectedLegNotSuccessful(job: "format", result: "failure"), which is correct. Ruled: land it, and each package fixes its own.Notes for review
ci-okkeeps its name.ci / matrix / ci-okis the only required status check on protected main; renaming it would silently un-gate the fleet. Verified against the ruleset, not assumed.ci.ymland one hop. Merging this is the cutover, since all callers track@main.actions: read+contents: read; asking for more fails the run at startup before any job is scheduled.install-system-deps,install-swift-sdkandconfigure-private-reposare restored to the tree. They were deleted while still in use, pinned to commits on main — which resolves, but left the workflow depending on files absent from the repository's own tree. All 25 in-repo action references now name one revision.ci.ymlcaller, so this PR has no checks of its own. The probes above are the evidence.