Skip to content

ci: improve workflow reliability and add multi-arch containers #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jgarzik merged 2 commits into from
Jan 19, 2026
Merged

ci: improve workflow reliability and add multi-arch containers #8

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7dfe984
ci: improve workflow reliability and add multi-arch containers
jgarzik Jan 19, 2026
405db6f
ci: address review feedback for workflow robustness
jgarzik Jan 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 12 additions & 18 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,22 @@ on:
pull_request:
branches: [main]

permissions:
contents: read

# Cancel in-progress runs for the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concurrency control uses github.ref which works well for branch pushes but may not be optimal for pull requests. For pull requests, github.ref is the merge ref (e.g., refs/pull/123/merge), which is unique per PR. However, when a PR is updated with new commits, the old CI run might still be valuable to cancel.

Consider using github.head_ref || github.ref which uses the PR head branch for pull requests, ensuring that new commits to the same PR cancel the old runs, while maintaining proper isolation for different PRs.

Suggested change
# Cancel in-progress runs for the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
# Cancel in-progress runs for the same branch or PR head
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}

Copilot uses AI. Check for mistakes.
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always

jobs:
# Fast checks first
lint:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
Expand All @@ -28,40 +37,25 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo build --release

# RooDB Integration Test Suite - runs all 4 configurations
# RooDB Integration Test Suite
roodb-tests:
needs: build
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2

# Install mysql client (used to connect to RooDB)
- name: Install mysql client (Ubuntu)
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get update && sudo apt-get install -y mysql-client

- name: Install mysql client (macOS)
if: matrix.os == 'macos-latest'
run: brew install mysql-client && echo "$(brew --prefix mysql-client)/bin" >> $GITHUB_PATH

- name: Install mysql client (Windows)
if: matrix.os == 'windows-latest'
run: choco install mysql-cli

# Run the full RooDB test suite
# On Linux: runs single_uring, single_posix, cluster_uring, cluster_posix
# On macOS/Windows: runs single_posix, cluster_posix (io_uring not available)
- name: Run RooDB test suite
run: cargo test --release
108 changes: 97 additions & 11 deletions .github/workflows/container.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,25 +1,55 @@
name: container

on:
# Build validation on PRs (no push)
pull_request:
push:
# Build and push only after CI passes on main/tags
workflow_run:
workflows: [CI]
types: [completed]
branches: [main]
# Also trigger on tags (CI doesn't run on tags, so direct trigger)
push:
tags: ['v*']
Comment on lines +7 to 13
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow_run trigger configuration only specifies branches: [main] but does not specify a filter for the workflow_run event's source. This means the workflow will only trigger when CI completes on the main branch, but pull requests to main will not trigger the container workflow via workflow_run (they will only trigger via the pull_request event).

Additionally, when CI runs on the main branch via a push event, the workflow_run will trigger, but since tags trigger the container workflow directly via push, and CI doesn't run on tags (as noted in the comment on line 11), there's an inconsistency in how the workflow is triggered for different event types.

Copilot uses AI. Check for mistakes.

permissions:
contents: read
packages: write

# Cancel in-progress runs for the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concurrency control uses github.ref which may cause issues with the workflow_run trigger. When a workflow is triggered by workflow_run, github.ref refers to the default branch (main) rather than the branch that triggered the original CI workflow. This means multiple workflow_run events from different branches would all share the same concurrency group and could cancel each other.

Consider using github.event.workflow_run.head_branch for workflow_run events to properly isolate concurrent runs by their source branch.

Suggested change
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_branch || github.ref }}

Copilot uses AI. Check for mistakes.
cancel-in-progress: true

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build:
runs-on: ubuntu-latest
# Skip if workflow_run triggered but CI failed
if: |
github.event_name == 'pull_request' ||
github.event_name == 'push' ||
(github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success')
runs-on: ${{ matrix.runner }}
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-latest
platform: linux/amd64
arch: amd64
- runner: ubuntu-24.04-arm
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runner label ubuntu-24.04-arm is being used for native ARM64 builds. This is a GitHub-hosted runner label that may not be available in all GitHub plans or repositories. You should verify that your repository has access to ARM64 runners before merging this change. If these runners are not available, the workflow will fail with a "No runner matching the specified labels was found" error.

For public repositories or those without access to GitHub-hosted ARM64 runners, consider using QEMU emulation on standard ubuntu-latest runners or self-hosted ARM64 runners instead.

Suggested change
- runner: ubuntu-24.04-arm
- runner: ubuntu-latest

Copilot uses AI. Check for mistakes.
platform: linux/arm64
arch: arm64
steps:
- name: Checkout
uses: actions/checkout@v4
with:
# For workflow_run, checkout the commit that triggered CI
ref: ${{ github.event.workflow_run.head_sha || github.ref }}
Comment on lines +51 to +52
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using workflow_run trigger with head_sha, there's a potential issue if the workflow_run is triggered by a pull request. The github.event.workflow_run.head_sha points to the PR commit, but the code doesn't check out the PR merge commit. This could result in building a container image from code that hasn't been tested with the target branch.

For pull_request events that triggered the CI workflow, you might want to use github.event.workflow_run.pull_requests[0].head.sha or ensure the ref matches what CI actually tested.

Suggested change
# For workflow_run, checkout the commit that triggered CI
ref: ${{ github.event.workflow_run.head_sha || github.ref }}
# For workflow_run, checkout the commit that CI tested
ref: ${{ (github.event_name == 'workflow_run' && github.event.workflow_run.pull_requests[0].head.sha) || github.event.workflow_run.head_sha || github.ref }}

Copilot uses AI. Check for mistakes.

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
Expand All @@ -32,6 +62,62 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata extraction in the build job (line 65-69) doesn't include any tag configuration, while the merge job (lines 121-131) has comprehensive tag configuration. This discrepancy means the build job generates labels but no tags, which is fine for the digest-based push approach. However, this could be clarified with a comment or by adding a tags field that explicitly outputs nothing, to make the intentional difference clear for maintainability.

Suggested change
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Intentionally omit tags in the build job; we push by digest only here.
tags: ""

Copilot uses AI. Check for mistakes.

- name: Build and push by digest
id: build
uses: docker/build-push-action@v6
with:
context: .
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' }}
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Docker build outputs specification on line 78 is very long and complex. The push parameter is conditionally set based on the event type, but it's embedded in a long outputs string. If this evaluates to push=false but push-by-digest=true is still set, the behavior might be unexpected.

Consider breaking this into separate parameters or adding a comment explaining the expected behavior for PR builds vs. production builds with push-by-digest.

Suggested change
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' }}
# For pull_request events we only build (no push). For workflow_run and tag
# pushes, we both build and push by digest. The `push` flag below is set
# conditionally based on the event type to reflect this behavior.
outputs: >-
type=image,
name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},
push-by-digest=true,
name-canonical=true,
push=${{ github.event_name != 'pull_request' }}

Copilot uses AI. Check for mistakes.
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,scope=${{ matrix.arch }},mode=max

- name: Export digest
if: github.event_name != 'pull_request'
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"

- name: Upload digest
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v4
with:
name: digests-${{ matrix.arch }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1

merge:
runs-on: ubuntu-latest
timeout-minutes: 10
if: github.event_name != 'pull_request'
needs: build
steps:
- name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-*
merge-multiple: true

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
Expand All @@ -44,12 +130,12 @@ jobs:
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shell command uses printf with a glob pattern * to iterate over digest files. However, if either architecture build fails (despite fail-fast: false), the merge job will still run but may have incomplete digests. This could result in creating a manifest list with only one architecture instead of both, which would be a silent failure.

Consider adding validation to ensure both architecture digests are present before creating the manifest, or handle the case where only partial builds succeeded.

Suggested change
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
set -euo pipefail
# Collect digest files and ensure we have one per architecture
shopt -s nullglob
digests=( * )
if [ "${#digests[@]}" -lt 2 ]; then
echo "Error: expected at least 2 digest files (for multiple architectures), found ${#digests[@]}." >&2
echo "Digest files present: ${digests[*]:-"<none>"}" >&2
exit 1
fi
digest_args=()
for d in "${digests[@]}"; do
digest_args+=( "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:${d}" )
done
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
"${digest_args[@]}"

Copilot uses AI. Check for mistakes.

- name: Inspect image
run: |
Comment on lines +155 to +156
Copy link

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The image inspection command uses steps.meta.outputs.version, but this output may not always contain a valid tag depending on the event type. For example, when triggered by workflow_run on the main branch, the metadata action will generate tags like edge (for main branch) or sha-xxx, and version might not be the most appropriate tag to inspect.

Consider using a more robust tag reference like ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:edge for main branch builds, or adjusting the inspection to use one of the generated tags that's guaranteed to exist.

Suggested change
- name: Inspect image
run: |
- name: Inspect image (workflow_run: edge tag)
if: github.event_name == 'workflow_run'
run: |
docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:edge
- name: Inspect image (push: version tag)
if: github.event_name == 'push'
run: |

Copilot uses AI. Check for mistakes.
docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}