Skip to content

Commit 09d9730

Browse files
committed
ci: stop cancelled Ring 1 runs from failing the Fly deploy
Two merges landing on main within a few minutes cancelled the earlier commit's User Journeys run (cancel-in-progress keyed on the ref), and fly-deploy's gate treats a cancelled reliability-ring1 as a hard failure — so a superseded commit produced a red deploy. - user-journeys.yml: keep superseded runs on main alive, matching docker.yml's expression. Every main commit's Ring 1 now reaches a conclusion of its own, which is what the gate reads. - fly-deploy.yml: when an upstream is red or cancelled for a commit main has already moved past, skip the deploy (deploy=false output) instead of failing. The newer commit's own run does the releasing. Also fold timed_out into the terminal conclusions the gate reacts to, rather than polling until the 65-minute job timeout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGkCEj3kPc4QMCxBPGo95p
1 parent 8cdf606 commit 09d9730

3 files changed

Lines changed: 50 additions & 12 deletions

File tree

.github/workflows/README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,12 @@ F2 wires this table into the actual gates:
104104
workflows' conclusion on the triggering commit before `deploy` runs —
105105
whichever of the two finishes second is the run that actually reaches
106106
`deploy` (the other is superseded by `fly-deploy`'s existing
107-
`cancel-in-progress` concurrency group).
107+
`cancel-in-progress` concurrency group). Because that gate reads a
108+
per-commit conclusion, `user-journeys.yml` does not cancel superseded runs
109+
on `main` — a run cancelled by the next merge would read as "Ring 1 failed"
110+
and block the release. And when the gate does see a red or cancelled
111+
upstream for a commit `main` has already moved past, it skips the deploy
112+
instead of failing: that commit's image is not what anyone is releasing.
108113
- **Ring 2**: `release.yaml` gained a per-OS "Reliability Ring 2
109114
packed-backend journey" step right after each OS's existing smoke-boot
110115
step, running linear-text-pipeline against that OS's packed backend

.github/workflows/fly-deploy.yml

Lines changed: 38 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,10 +58,12 @@ jobs:
5858
timeout-minutes: 65
5959
outputs:
6060
sha: ${{ github.event.workflow_run.head_sha }}
61+
deploy: ${{ steps.poll.outputs.deploy }}
6162
env:
6263
GH_TOKEN: ${{ github.token }}
6364
steps:
6465
- name: Poll for both workflows' success on this commit
66+
id: poll
6567
shell: bash
6668
run: |
6769
set -euo pipefail
@@ -74,26 +76,50 @@ jobs:
7476
--jq '.workflow_runs[0].conclusion // "pending"'
7577
}
7678
79+
# True once a newer commit has landed on main. That commit's own
80+
# fly-deploy run releases the newer image, so anything this run could
81+
# still conclude about ${sha} is moot — and a red X here would be
82+
# noise, not a broken deploy.
83+
superseded() {
84+
[ "$(gh api "repos/${repo}/commits/main" --jq .sha)" != "${sha}" ]
85+
}
86+
87+
skip() {
88+
echo "::notice::${1} Superseded on main; skipping the deploy of ${sha}."
89+
echo "deploy=false" >> "$GITHUB_OUTPUT"
90+
exit 0
91+
}
92+
7793
for attempt in $(seq 1 60); do
7894
docker_conclusion="$(conclusion_for docker.yml)"
7995
journeys_conclusion="$(conclusion_for user-journeys.yml)"
8096
echo "attempt ${attempt}/60: docker.yml=${docker_conclusion} user-journeys.yml=${journeys_conclusion}"
8197
8298
if [ "${docker_conclusion}" = "success" ] && [ "${journeys_conclusion}" = "success" ]; then
8399
echo "Both workflows succeeded for ${sha}."
100+
echo "deploy=true" >> "$GITHUB_OUTPUT"
84101
exit 0
85102
fi
86-
if [ "${docker_conclusion}" = "failure" ] || [ "${docker_conclusion}" = "cancelled" ]; then
87-
echo "::error::docker.yml concluded '${docker_conclusion}' for ${sha}; not deploying."
88-
exit 1
89-
fi
90-
if [ "${journeys_conclusion}" = "failure" ] || [ "${journeys_conclusion}" = "cancelled" ]; then
91-
echo "::error::user-journeys.yml (reliability-ring1) concluded '${journeys_conclusion}' for ${sha}; not deploying."
92-
exit 1
93-
fi
103+
case "${docker_conclusion}" in
104+
failure|cancelled|timed_out)
105+
if superseded; then skip "docker.yml concluded '${docker_conclusion}' for ${sha}."; fi
106+
echo "::error::docker.yml concluded '${docker_conclusion}' for ${sha}; not deploying."
107+
exit 1
108+
;;
109+
esac
110+
case "${journeys_conclusion}" in
111+
failure|cancelled|timed_out)
112+
if superseded; then skip "user-journeys.yml (reliability-ring1) concluded '${journeys_conclusion}' for ${sha}."; fi
113+
echo "::error::user-journeys.yml (reliability-ring1) concluded '${journeys_conclusion}' for ${sha}; not deploying."
114+
exit 1
115+
;;
116+
esac
94117
sleep 60
95118
done
96119
120+
if superseded; then
121+
skip "Timed out waiting for docker.yml and user-journeys.yml on ${sha}."
122+
fi
97123
echo "::error::Timed out waiting for docker.yml and user-journeys.yml to both complete for ${sha}."
98124
exit 1
99125
@@ -103,11 +129,13 @@ jobs:
103129
runs-on: ubuntu-latest
104130
# Skip when the image build (or the reliability Ring 1 gate) failed/was
105131
# cancelled. workflow_dispatch always runs (manual re-deploy); on
106-
# workflow_run, `gate` must have completed successfully.
132+
# workflow_run, `gate` must have completed successfully AND still consider
133+
# this commit worth releasing — it reports deploy=false for a commit main
134+
# has already moved past, whose own run does the releasing.
107135
if: >
108136
always() &&
109137
(github.event_name == 'workflow_dispatch' ||
110-
needs.gate.result == 'success')
138+
(needs.gate.result == 'success' && needs.gate.outputs.deploy == 'true'))
111139
steps:
112140
- uses: actions/checkout@v7
113141
with:

.github/workflows/user-journeys.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,14 @@ on:
2727
permissions:
2828
contents: read
2929

30+
# On main, every commit's reliability-ring1 run must reach a conclusion of its
31+
# own: fly-deploy.yml gates the deploy on this workflow's result for that exact
32+
# head_sha, so a run cancelled by a follow-up merge reads as "Ring 1 failed" and
33+
# blocks the release. Same reasoning (and same expression) as docker.yml.
34+
# Elsewhere — PR branches, tags — superseded runs are still cancelled.
3035
concurrency:
3136
group: ${{ github.workflow }}-${{ github.ref }}
32-
cancel-in-progress: true
37+
cancel-in-progress: ${{ github.ref != 'refs/heads/main' && !startsWith(github.ref, 'refs/tags/') }}
3338

3439
jobs:
3540
journeys:

0 commit comments

Comments
 (0)