Feat/incus vm support#48
Draft
tejashgawasibm wants to merge 7 commits into
Draft
Conversation
Add Incus container build support as an alternative to LXD, maintaining API compatibility while supporting the community-led Incus fork. Core Components: - Incus installation scripts for Ubuntu and CentOS (builds from source) - incus_container.sh build script with LXD-compatible workflow - Preseed configurations for ppc64le, s390x, x86_64 architectures - Flexible version configuration via environment variables (default: INCUS_VERSION=v7.1.0, RAFT_VERSION=v0.22.1) - Configurable LVM storage with loop device support - Idempotent initialization for safe re-runs Features: - Multi-architecture support (ppc64le, s390x, x86_64) - Automatic Incus installation if not present - LVM thin provisioning with configurable pool size - Network isolation with dedicated bridge (incusbr0) - Compatible with distrobuilder-generated images Tested on: - Host: CentOS Stream 10 (ppc64le) - Incus: v7.1 - Container: Ubuntu 24.04 LTS Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
…un.sh Integration: - Add Incus Container option to run.sh interactive menu - Add Incus-specific command-line flags (--incus-debug, --skip-incus-*) - Add Incus variables to setup_vars.sh for build customization Distrobuilder Compatibility Fixes: - Support both deb822 (ubuntu.sources) and classic (sources.list) APT formats - Remove manual netplan configuration (distrobuilder images pre-configured) - Add software-properties-common to vital packages for add-apt-repository - Fix script execution order: install git before runner-package build Architecture-Specific Fixes (ppc64le): - Add graceful error handling for Apache installation (not available on ppc64le) - Add non-fatal error handling for Ansible/pipx package installations - Add runner test failure workaround (tests fail but binary works) - Add defensive checks for optional tools (yarn, npm, needrestart) Build Reliability Improvements: - Add Maven download fallback to archive.apache.org - Add tar export cleanup to prevent read-only file conflicts - Replace Incus preseed with idempotent imperative commands - Fix group membership to conditionally add docker/incus groups These fixes ensure compatibility with distrobuilder-generated images and handle architecture-specific issues on ppc64le, s390x, and x86_64. Tested on: - Host: CentOS Stream 10 (ppc64le) - Container: Ubuntu 24.04 (complete setup) - Incus: v7.1 Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
Add comprehensive Incus container/VM support with the following features: - Hybrid preseed/manual initialization approach * Try preseed first on fresh installations (no storage/network) * Fallback to manual configuration for partial states or preseed failures * Maintains idempotency for re-runs and partial configurations - Address PR review suggestions: * Reduce LVM storage size from 180GiB to 100GiB for boot volume compatibility * Use helper functions (install_dpkgs, install_dnfpkgs, update_dpkgs) for package management * Add image type filter to incus_container.sh to distinguish containers from VMs - Sync with upstream actions/runner-images: * Pin sbt to 1.x versions to avoid compatibility issues - Code quality improvements: * Replace 'cat file | command' with 'command < file' (ShellCheck SC2002) * Fix network existence check logic Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
…tainer - Auto-select image import method based on host architecture - Use Incus image server for x86_64/aarch64 (fast, ~30s) - Use Distrobuilder for ppc64le/s390x (custom build, ~10-15min) - Simplify the Ubuntu image import workflow Helper Scripts: - import_ubuntu_base_images.sh: Main wrapper with auto-detection * Detects architecture and routes to appropriate method * Can be sourced by incus_container.sh, incus_vm.sh, etc. * Reusable across different Incus deployment types - import-incus-ubuntu-image.sh: Incus image server import * For x86_64/aarch64 architectures - build-distrobuilder-image.sh: Custom image builder * For ppc64le/s390x architectures Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
Add scripts/incus_vm.sh to build GitHub Actions runner VM images using Incus, and wire the VM and container workflows into run.sh. scripts/incus_vm.sh: - Uses incus init --vm + configure + incus start instead of launch so CPU and memory limits are set before the VM first boots - configure_cpu_resources(): pins VM to available host CPUs by reading currently used pinned CPUs from running instances and allocating from the remainder; prevents CPU contention - configure_memory_resources(): reads /proc/meminfo, reserves a 512 MiB host buffer, aligns to 256 MiB boundary required by QEMU on ppc64le (misaligned sizes cause qemu-system-ppc64 error) - Expands root partition on /dev/sda2 (not sda1 which is the PReP boot partition on ppc64le) via growpart + resize2fs after reboot - Image aliases carry -vm suffix (e.g. ubuntu-24.04-ppc64le-vm) to distinguish from container images in the same Incus store - Published with --compression gzip to reduce storage for large QEMU disk images run.sh: add Incus VM and Incus Container to the interactive menu scripts/incus_container.sh: honour --skip-snap-lxd flag in ensure_incus(); add OS family mapping for Fedora/RHEL/AlmaLinux scripts/helpers/import_ubuntu_base_images.sh: route vm image type to distrobuilder for all architectures (VM images are not published on the Incus image server) scripts/helpers/import-incus-ubuntu-image.sh: minor cleanup scripts/lxd_container.sh, scripts/lxd_vm.sh: map Fedora/RHEL/ AlmaLinux/Rocky to CentOS build scripts and Debian to Ubuntu so all host OS families resolve the correct installer path Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
Apply a series of reliability fixes to both images/centos/scripts/
build/install-incus.sh and images/ubuntu/scripts/build/install-incus.sh
so each script is fully idempotent across partial failures and
repeated runs.
Daemon startup:
- Create incus and incus-admin groups before starting incusd;
without them --group incus-admin causes daemon start to fail
- Force-kill any stale incusd process (pkill -9) and remove all
three stale socket locations before each daemon start:
/run/incus/unix.socket
/var/run/incus/unix.socket
/var/lib/incus/unix.socket
- Use mktemp for the incusd log file to avoid permission errors
when the log was created by a prior run as a different user
- Use pgrep -x incusd instead of kill -0 $PID to detect crashes,
because incusd may fork internally after the initial launch
- Replace blind sleep with a 30-iteration poll loop using
incus admin waitready --timeout=1 as the readiness check,
never checking the socket path directly
- Clean up stale incusbr0 bridge interface before network create
to make the manual fallback path idempotent
LVM setup:
- Run wipefs -af and pvremove -ff -y before pvcreate to clear
any stale LVM/filesystem signatures left by a prior partial run
- Clean up stale loop devices with losetup -j before pvcreate
Preseed init:
- Re-query STORAGE_EXISTS and NETWORK_EXISTS immediately after
preseed failure so the manual fallback path operates on actual
current state, not the state captured before preseed ran
Profile configuration (ppc64le only):
- Set limits.memory=16GiB and raw.qemu=-m 16384M,slots=0,maxmem=16384M
on the default profile; raw.qemu is required on POWER because QEMU
enforces a 256 MiB memory-alignment constraint
Preseed YAML updates:
- incus_init_host_ppc64le.yml: add limits.memory and raw.qemu to
profile config so the preseed path also applies the constraint
- incus_init_host_s390x.yml: minor trailing-newline cleanup
- incus_init_container_{ppc64le,s390x,x86_64}.yml: trailing-newline
formatting cleanup
Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
Member
Author
|
Dependent on # (feat/incus — Incus container support). This PR must not be merged until that one lands in main. |
tejashgawasibm
marked this pull request as draft
July 17, 2026 06:28
scripts/helpers/build-distrobuilder-image.sh: - Use BASH_SOURCE[0] instead of \$0 for REPO_ROOT resolution so patch paths resolve correctly when the script is sourced - Apply patches/distrobuilder.patch and patches/lxc-ci.patch to the cloned distrobuilder and ubuntu.yaml before building - Switch APT mirror from HTTP to HTTPS to prevent ECONNRESET during package downloads inside the distrobuilder chroot - Probe the FAU mirror (ftp.fau.de/ubuntu-ports) before the build; use it as primary and fall back to ports.ubuntu.com if unreachable - Fix false success: capture PIPESTATUS[0] after the distrobuilder pipeline so a non-zero exit from distrobuilder is not masked by tee - Support --vm flag: pass --vm to distrobuilder and import the resulting incus.tar.xz + disk.qcow2 pair instead of rootfs patches/distrobuilder.patch (IBM platform patch for distrobuilder): - ppc64le: 8 MiB PReP boot partition (GPT type 4100) instead of 100 MiB EFI partition; skip mkfs.vfat and /boot/efi mount - ppc64le: export DISTROBUILDER_LOOP_DEVICE env var into chroot so post-files hooks can install GRUB to the PReP partition - s390x: 500 MiB ext4 boot partition mounted at /boot; use zipl as the bootloader - fstab generator: 3-way arch switch (ppc64le = no entry, s390x = ext4 /boot, default = vfat /boot/efi) - Remove mkfs.vfat from hard dependency check since ppc64le and s390x do not use it patches/lxc-ci.patch (IBM platform patch for ubuntu.yaml): - ppc64le: install grub-ieee1275; add post-files action that writes device.map, creates /dev/nvram if missing, and installs GRUB to the PReP partition with target=powerpc-ieee1275 - s390x: install s390-tools; add post-files action that generates /etc/zipl.conf and runs zipl to install the bootloader - Restrict existing x86/arm64 update-grub action to amd64 + arm64 - Replace hardcoded enp5s0 NIC name with id0 match: name: "en*" so VMs boot correctly on all architectures (replaces the removed patch_netplan_config() Python function that did the same at runtime) Both patch files are plain unified diffs (no mail headers) starting directly at the first diff --git line. Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
tejashgawasibm
force-pushed
the
feat/incus-vm-support
branch
from
July 17, 2026 06:31
9ca3836 to
4982bdd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Adds Incus VM image build support on top of the existing Incus container support. Also hardens the Incus installer and includes patches for distrobuilder and lxc-ci.
Changes
Incus VM Support
incus_vm.shfor VM image builds.run.sh.Installer Improvements
install-incus.shreliability and idempotency.limits.memoryandraw.qemu.Patches
Testing