Skip to content

[Enhancement] Run x86_64 functional tests inside QEMU VM (reproducible environment for NixOS/downstream packagers) #434

Description

@mcdope

Background

The existing native functional test suite (Build & test / DoMagicOnConfiguredCustomRunner) runs directly on the CI runner host. This works for our own CI but creates a significant barrier for downstream packagers — particularly NixOS packagers, who use a read-only Nix store and cannot reproduce the test environment without significant manual setup.

The multi-arch QEMU infrastructure introduced in #429 (arm64, armhf, ppc64el, riscv64) proves the pattern works well: a pre-provisioned golden image provides a clean, reproducible Ubuntu Jammy environment; tests run inside it; the image is version-controlled via PROVISION_VERSION. Extending this to x86_64 (amd64) would give any packager a single command to reproduce the full functional test suite locally.


Goal

Add an amd64 QEMU full-system test path that:

  1. Uses the same golden image strategy as the ARM/ppc64el/riscv64 paths
  2. Runs the full tests/can-actually-be-used/run-tests.sh suite inside a clean Ubuntu 22.04 VM
  3. Is invocable locally (make provision-qemu-images + run-tests-in-qemu.sh amd64 <deb>)
  4. Runs in CI as a new FunctionalTest-amd64 job, gated on the existing Gate job

Technical notes

Arch naming: amd64 — consistent with Debian/Ubuntu naming used by all other arches in run-tests-in-qemu.sh.

Cloud image: https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img

QEMU binary: qemu-system-x86_64 (from qemu-system-x86 package — already available on the CI runner).

Firmware: UEFI via OVMF (ovmf package: /usr/share/OVMF/OVMF_CODE.fd). Use the existing find_bios helper in run-tests-in-qemu.sh with the known OVMF paths.

Machine/CPU:

QEMU_MACHINE="-M q35 -cpu max -smp 2 -m 2048"

q35 is the modern x86 chipset; -cpu max enables all available TCG features for best compatibility. The CI runner does not have KVM available, so TCG emulation is used — same as ppc64el and riscv64.

VirtIO bus: PCI (x86 q35 machine uses PCI, same as ppc64el pseries):

QEMU_BLK_DEV="virtio-blk-pci"
QEMU_NET_DEV="virtio-net-pci"

Implementation steps

1. tests/can-actually-be-used/run-tests-in-qemu.sh

Add amd64) case to the case "$ARCH" block:

amd64)
    QEMU_BIN="qemu-system-x86_64"
    IMAGE_URL="https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"
    IMAGE_CACHE="${CACHE_DIR}/jammy-amd64.img"
    QEMU_MACHINE="-M q35 -cpu max -smp 2 -m 2048"
    BIOS_PATH="$(find_bios \
        /usr/share/OVMF/OVMF_CODE.fd \
        /usr/share/ovmf/OVMF.fd)" || true
    if [ -z "$BIOS_PATH" ]; then
        echo "Error: cannot find x86_64 OVMF firmware" >&2; exit 1
    fi
    QEMU_BIOS="-bios ${BIOS_PATH}"
    QEMU_BLK_DEV="virtio-blk-pci"
    QEMU_NET_DEV="virtio-net-pci"
    ;;

Update header comments and the unsupported-arch error message.

2. Makefile

  • Add build-debian-amd64 target (can alias build-debian since the native arch is already amd64)
  • Add amd64 to provision-qemu-images as a fifth parallel job
  • No new LIBDIR entry needed (x86_64 is already handled)

3. .github/workflows/build-and-test-cross-arch.yml

Add FunctionalTest-amd64 job mirroring the other FunctionalTest jobs:

  • Golden image check (file: jammy-amd64-provisioned-v${PROVISION_VERSION}.qcow2)
  • QEMU prereqs: sudo apt-get install -y qemu-system-x86 ovmf qemu-utils cloud-image-utils
  • Build step: make build-debian (native, already produces the amd64 .deb)
  • Run: run-tests-in-qemu.sh amd64 $(ls .build/libpam-usb_*_amd64.deb | head -1)
  • Cleanup gated on check_image_amd64.outcome == 'success'
  • Add needs: [Gate]

4. Provisioning

After implementation, run on the CI runner:

make provision-qemu-images

This will provision arm64, armhf, ppc64el, riscv64, and amd64 in parallel.
Bump PROVISION_VERSION when changing the provisioned environment.


Acceptance criteria

  • make provision-qemu-images successfully provisions a jammy-amd64-provisioned-vN.qcow2
  • tests/can-actually-be-used/run-tests-in-qemu.sh amd64 .build/libpam-usb_*_amd64.deb passes all three filesystem iterations (vfat, ext4, exfat)
  • FunctionalTest-amd64 CI job passes green
  • Any NixOS packager can run make provision-qemu-images + the test script on their own machine to reproduce the CI result

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions