Implement a subcommand to output SUT definitions

[bastelfreak]

Right now, beaker generates and starts different SUTs - systems under test. The rough workflow:

matrix generation workflow

• A module defines supported openvox and or puppet versions in metadata.json
• Based on that, we know the different major versions, e.g. puppet7, puppet8, openvox7, openvox8
• A module also lists supported operating system major versions
• puppet_metadata has a list of EOL dates for each major OS
• puppet_metadata also knows for which major OS version a puppet/openvox package is available
• based on that information, puppet_metadata provides a tool called metadata2gha which can generate a CI matrix based on supported major puppet/openvox versions multiplied by each supported, and not EOL, major OS version
• the command is bundle exec metadata2gha. It outputs the matrix for unit jobs, puppet_unit_test_matrix, and for the acceptance tests, puppet_beaker_test_matrix.

puppet_unit_test_matrix=[{"puppet":8,"ruby":"3.2"}]
puppet_beaker_test_matrix=[{"name":"OpenVox 8 - AlmaLinux 8","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"almalinux8-64{hostname=almalinux8-64-openvox8}"}},{"name":"OpenVox 8 - AlmaLinux 9","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"almalinux9-64{hostname=almalinux9-64-openvox8}"}},{"name":"Distro Puppet - Archlinux rolling","env":{"BEAKER_PUPPET_COLLECTION":"none","BEAKER_SETFILE":"archlinuxrolling-64"}},{"name":"OpenVox 8 - CentOS 9","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"centos9-64{hostname=centos9-64-openvox8}"}},{"name":"OpenVox 8 - Debian 11","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"debian11-64{hostname=debian11-64-openvox8}"}},{"name":"OpenVox 8 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-openvox8}"}},{"name":"OpenVox 8 - Debian 13","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"debian13-64{hostname=debian13-64-openvox8}"}},{"name":"OpenVox 8 - OracleLinux 8","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"oracle8-64{hostname=oracle8-64-openvox8}"}},{"name":"OpenVox 8 - OracleLinux 9","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"oracle9-64{hostname=oracle9-64-openvox8}"}},{"name":"OpenVox 8 - Rocky 8","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"rocky8-64{hostname=rocky8-64-openvox8}"}},{"name":"OpenVox 8 - Rocky 9","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"rocky9-64{hostname=rocky9-64-openvox8}"}},{"name":"OpenVox 8 - Ubuntu 22.04","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"ubuntu2204-64{hostname=ubuntu2204-64-openvox8}"}},{"name":"OpenVox 8 - Ubuntu 24.04","env":{"BEAKER_PUPPET_COLLECTION":"openvox8","BEAKER_SETFILE":"ubuntu2404-64{hostname=ubuntu2404-64-openvox8}"}}]

A simplified and pretty-printed version:

[
  {
    "name": "OpenVox 8 - AlmaLinux 8",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "openvox8",
      "BEAKER_SETFILE": "almalinux8-64{hostname=almalinux8-64-openvox8}"
    }
  },
  {
    "name": "OpenVox 8 - AlmaLinux 9",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "openvox8",
      "BEAKER_SETFILE": "almalinux9-64{hostname=almalinux9-64-openvox8}"
    }
  },
  {
    "name": "Distro Puppet - Archlinux rolling",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "none",
      "BEAKER_SETFILE": "archlinuxrolling-64"
    }
  }
]

• It's defined in our repo for shared workflows, gha-puppet.
• We will then create one GitHub job for each element in the matrix, basically running beaker in parallel and executing the same set of acceptance tests for various operating systems.
• Each job contains an environment variable for the beaker setfile, named BEAKER_SETFILE.
• beaker supports various plugins, one is beaker-docker. It generates various Containerfiles on the fly and posts them to the docker/podman API to create & start a container
• A setfile defines all systems for a SUT suite, a simple example is ubuntu2404-64{hostname=ubuntu2404-64-openvox8}
• This will spin up a single Ubuntu 24.04 system with AMD64 architecture and set a hostname, the default hypervisor will be used (usually docker or podman)
• This is rendered into a setfile via beaker-hostgenerator: bundle exec beaker-hostgenerator 'ubuntu2204-64{hostname=ubuntu2404-64-openvox8}' --hypervisor docker

---
HOSTS:
  ubuntu2404-64-openvox8:
    docker_cmd:
    - "/sbin/init"
    docker_image_commands:
    - cp /bin/true /sbin/agetty
    - apt-get install -y net-tools wget locales iproute2 gnupg
    - locale-gen en_US.UTF-8
    - echo LANG=en_US.UTF-8 > /etc/default/locale
    image: amd64/ubuntu:24.04
    platform: ubuntu-24.04-amd64
    hypervisor: docker
    roles:
    - agent
CONFIG: {}

• This information will be forwarded to the beaker plugin, beaker-docker in this case, to generate a Containerfile

Current challenges with acceptance tests

We have two issues:

• Most container images are pulled from dockerhub, and that has annoying rate limits that break our CI. Restarting the jobs is very annoying and time consuming.
• During each container start, we install various packages, which just wastes time

Proposed solution

In voxpupuli/beaker-docker@e13be08, @ekohl added a new subcommand to beaker-docker. It allows us to generate a Containerfile and place it into the local filesystem

bundle exec beaker-docker containerfile centos9-64

gives us:

FROM quay.io/centos/centos:stream9
ENV container docker
RUN dnf clean all && dnf install -y sudo openssh-server openssh-clients iputils && ssh-keygen -A && sed 's@session *required *pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/*
RUN mkdir -p /var/run/sshd && echo root:root | chpasswd
RUN sed -ri -e 's/^#?PermitRootLogin .*/PermitRootLogin yes/' -e 's/^#?PasswordAuthentication .*/PasswordAuthentication yes/' -e 's/^#?UseDNS .*/UseDNS no/' -e 's/^#?UsePAM .*/UsePAM no/' -e 's/^#?MaxAuthTries.*/MaxAuthTries 1000/' /etc/ssh/sshd_config $(compgen -G '/etc/ssh/sshd_config.d/*' || echo '')
RUN cp /bin/true /sbin/agetty
RUN dnf install -y cronie crontabs initscripts iproute openssl wget which glibc-langpack-en hostname
EXPOSE 22
CMD ["/sbin/init"]

• build a CI in https://github.com/voxpupuli/beaker-container-images that generates a matrix for all supported OSes per openvox version
• Generate the Containerfile
• extend it to add the openvox7 / openvox8 repo
• build the container
• upload and tag it

Afterwards we need some new logic in beaker(-docker) to use our own images for actual CI runs

[bastelfreak]

First step is done, with voxpupuli/beaker-container-images#13 we now can build containers for each OS in metadata.json.