Skip to content

Commit 8fc9b68

Browse files
avalleteclaudejgoux
authored
chore(ci): add setup-cli smoke test workflow (#5315)
Add a new GitHub Actions workflow to smoke test the `supabase/setup-cli` action against published CLI versions. This catches regressions like setup-cli#427 where musl libc and archive layout changes broke the action on Alpine silently. **Key changes:** - New `.github/workflows/setup-cli-smoke-test.yml` workflow that: - Tests CLI installation across Ubuntu, macOS, and Windows runners - Includes a dedicated Alpine container job to catch musl libc regressions - Verifies the installed CLI version matches the expected version - Can be triggered manually via `workflow_dispatch` or called from other workflows - Updated `.github/workflows/release-shared.yml` to: - Add a `setup-cli-smoke` job that runs after beta releases - Runs as a post-publish signal without gating other release jobs - Uses `always() && needs.publish.result == 'success'` to run even when homebrew/scoop jobs are skipped or fail The Alpine job is kept separate because GitHub Actions only honors the `container:` field on Linux runners, and the musl-vs-glibc regressions only reproduce inside a real Alpine container. Closes: CLI-1507 https://claude.ai/code/session_01NzzeYET8sYhpCVdSvBBJhC --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Julien Goux <hi@jgoux.dev>
1 parent 7a0deab commit 8fc9b68

2 files changed

Lines changed: 135 additions & 0 deletions

File tree

‎.github/workflows/release-shared.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,3 +415,22 @@ jobs:
415415
run: pnpm exec bun apps/cli/scripts/update-scoop.ts --version "${VERSION}" --name "${SCOOP_NAME}"
416416
env:
417417
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
418+
419+
# Post-publish smoke test for the `supabase/setup-cli` GitHub Action against
420+
# the just-released CLI. Runs last and intentionally does not gate
421+
# publish-homebrew / publish-scoop — by the time the smoke runs, the npm
422+
# package and GitHub release are already live and the brew/scoop pushes
423+
# have either succeeded or skipped, so a setup-cli regression surfaces as
424+
# a red post-release signal without holding back the rest of the channel.
425+
#
426+
# Depends on the publish jobs only via `needs` for ordering; the `if`
427+
# uses `always() && needs.publish.result == 'success'` so the smoke still
428+
# runs when publish-homebrew / publish-scoop are skipped (alpha) or fail.
429+
# The reusable workflow can also be dispatched manually against any
430+
# already-published version when debugging setup-cli regressions.
431+
setup-cli-smoke:
432+
needs: [publish, publish-homebrew, publish-scoop]
433+
if: ${{ always() && !inputs.dry_run && needs.publish.result == 'success' }}
434+
uses: ./.github/workflows/setup-cli-smoke-test.yml
435+
with:
436+
version: ${{ inputs.version }}

‎.github/workflows/setup-cli-smoke-test.yml‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
name: setup-cli Smoke Test
2+
3+
# Smoke test for the `supabase/setup-cli` GitHub Action against a specific
4+
# published CLI version. Run automatically after every beta release (called
5+
# from release-shared.yml's `setup-cli-smoke` job) and also manually via
6+
# workflow_dispatch when debugging setup-cli regressions or verifying that
7+
# a previously broken environment (e.g. Alpine) is back to green.
8+
#
9+
# Exists primarily to catch regressions like supabase/setup-cli#427 where
10+
# musl libc + archive layout changes broke the action on Alpine silently.
11+
12+
on:
13+
workflow_call:
14+
inputs:
15+
version:
16+
description: Supabase CLI version to install via setup-cli (must already be published to GitHub Releases)
17+
required: true
18+
type: string
19+
workflow_dispatch:
20+
inputs:
21+
version:
22+
description: Supabase CLI version to install via setup-cli (must already be published to GitHub Releases)
23+
required: true
24+
type: string
25+
26+
permissions:
27+
contents: read
28+
29+
jobs:
30+
setup-cli-smoke-test:
31+
name: setup-cli ${{ matrix.major-version }} (${{ matrix.runner }})
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
runner:
36+
- ubuntu-latest
37+
- macos-latest
38+
- windows-latest
39+
# GitHub Actions doesn't allow expressions in `uses:`, so the action
40+
# ref is selected via two `if:`-gated install steps below rather than
41+
# interpolating `matrix.major-version` into a single `uses:` line.
42+
major-version:
43+
- v1
44+
- v2
45+
runs-on: ${{ matrix.runner }}
46+
env:
47+
VERSION: ${{ inputs.version }}
48+
steps:
49+
- name: Install Supabase CLI via setup-cli@v1
50+
if: matrix.major-version == 'v1'
51+
uses: supabase/setup-cli@v1
52+
with:
53+
version: ${{ inputs.version }}
54+
- name: Install Supabase CLI via setup-cli@v2
55+
if: matrix.major-version == 'v2'
56+
uses: supabase/setup-cli@v2
57+
with:
58+
version: ${{ inputs.version }}
59+
- name: Verify supabase --version matches the expected version
60+
shell: bash
61+
run: |
62+
set -euo pipefail
63+
actual="$(supabase --version | tr -d '\r' | head -n1)"
64+
echo "supabase --version: ${actual}"
65+
if [ "${actual}" != "${VERSION}" ]; then
66+
echo "Version mismatch: expected ${VERSION}, got ${actual}" >&2
67+
exit 1
68+
fi
69+
70+
# Alpine leg of the setup-cli smoke test. Kept as a separate job because
71+
# GitHub Actions only honours the `container:` field on Linux runners, and
72+
# mixing container + non-container entries in a single matrix via
73+
# `container: ${{ matrix.container }}` is fragile when the value is empty.
74+
#
75+
# The musl-vs-glibc + archive layout regressions tracked in
76+
# supabase/setup-cli#427 only reproduce inside a real Alpine container, so
77+
# this job is the actual signal the workflow was created for.
78+
setup-cli-smoke-test-alpine:
79+
name: setup-cli ${{ matrix.major-version }} (alpine)
80+
strategy:
81+
fail-fast: false
82+
matrix:
83+
major-version:
84+
- v1
85+
- v2
86+
runs-on: ubuntu-latest
87+
# `node:20-alpine` ships a musl-linked Node, which is required for
88+
# JavaScript-based actions (setup-cli included) to launch inside an
89+
# Alpine container — the runner's mounted glibc Node won't execute here.
90+
container:
91+
image: node:20-alpine
92+
env:
93+
VERSION: ${{ inputs.version }}
94+
steps:
95+
- name: Install Alpine prerequisites
96+
run: apk add --no-cache bash curl tar
97+
- name: Install Supabase CLI via setup-cli@v1
98+
if: matrix.major-version == 'v1'
99+
uses: supabase/setup-cli@v1
100+
with:
101+
version: ${{ inputs.version }}
102+
- name: Install Supabase CLI via setup-cli@v2
103+
if: matrix.major-version == 'v2'
104+
uses: supabase/setup-cli@v2
105+
with:
106+
version: ${{ inputs.version }}
107+
- name: Verify supabase --version matches the expected version
108+
shell: bash
109+
run: |
110+
set -euo pipefail
111+
actual="$(supabase --version | tr -d '\r' | head -n1)"
112+
echo "supabase --version: ${actual}"
113+
if [ "${actual}" != "${VERSION}" ]; then
114+
echo "Version mismatch: expected ${VERSION}, got ${actual}" >&2
115+
exit 1
116+
fi

0 commit comments

Comments
 (0)