Default test matrix moved from ubuntu-24.04-arm and ubuntu-slim to ubuntu-26.04-arm and ubuntu-26.04

Author: kdeldycke

changelog.md

@@ -9,6 +9,7 @@
 - **Breaking:** the `update-dep-graph` job moved from `autofix.yaml` into the release engine, its only firing moment being a release push. A required check or `needs:` edge naming it under Autofix must follow.
 - **Breaking:** manual version-bump commits now read `` [changelog] Bump minor version to `vX.Y.0` ``: every version-machinery commit carries the `[changelog] ` prefix, and anything matching the old unprefixed titles must follow.
 - New `lint-deps` command and release-lane job, blocking a release whose dependencies do not all resolve from PyPI: git branches, forks, local paths, direct URLs and private indexes. Exempt a package with `[tool.repomatic] lint-deps.allow`.
+- **Breaking:** the default test matrix moved from `ubuntu-24.04-arm` and `ubuntu-slim` to `ubuntu-26.04-arm` and `ubuntu-26.04`, which also moves the PR Linux slot and the `3.14t` smoke test. Pin the old images back with `test-matrix.replace.os = { "ubuntu-26.04-arm" = "ubuntu-24.04-arm" }`.
 - New `runner-images` job and CLI command, opening an issue that lists GitHub's open runner-image announcements and flags the ones retiring an image the repo runs on. Opt out with `runner-images = false`.
 - Autofix jobs no longer run on version-bump pushes: the 16 remaining jobs only re-checked machine-generated, ship-gated commits, and drift stays covered by the next push and the weekly sweep.
 - The tool runner now retries a download up to 3 times on transient network failures, instead of failing the job on a one-off TLS or truncation error.

claude.md

@@ -390,7 +390,8 @@ When releasing `kdeldycke/repomatic`, see [`docs/upstream-development.md` § Rel
 
 - **Cover the shipped config broadly; probe unreleased axes narrowly; smoke-test released flavors.** Released dependencies on stable Python get the full cross-platform spread. Unreleased dependency branches and prerelease Python run on one runner as `continue-on-error` probes (`test-matrix.unstable`), never across platforms. A released free-threaded build (`3.14t`) runs **stable** on a single runner (a `python-version` variation pinned with `exclude`, left out of `unstable`), not as an `unstable` probe.
 - **Pin the dependency floor, and any release a workaround targets.** Add the floor of a supported range as an explicit matrix value, plus any mid-range release a shim works around: that is the version that catches the shim regressing.
-- **Select runners by measured speed and workload, not architecture** (read your own CI timings; the `docs/test-matrix.md` inventory has the numbers). The parallel `pytest --numprocesses=auto` suite favors `ubuntu-24.04-arm` (the test PR Linux slot), while setup-bound light jobs keep the lean `ubuntu-slim`. Where one fast runner suffices, `ubuntu-24.04-arm` is the default (fastest and cheapest tier; hosted macOS bills ~10x Linux), so `macos-26`/Windows are reserved for the OS coverage only they add. Drop the slower twin of an OS pair via `test-matrix.remove.os`.
+- **Select runners by measured speed and workload, not architecture** (read your own CI timings; the `docs/test-matrix.md` inventory has the numbers). The parallel `pytest --numprocesses=auto` suite favors `ubuntu-26.04-arm` (the test PR Linux slot), while setup-bound light jobs keep the lean `ubuntu-slim`. Where one fast runner suffices, `ubuntu-26.04-arm` is the default (fastest and cheapest tier; hosted macOS bills ~10x Linux), so `macos-26`/Windows are reserved for the OS coverage only they add. Drop the slower twin of an OS pair via `test-matrix.remove.os`.
+- **An image is stable once validated here, not once GitHub relabels it.** The Ubuntu 26.04 axes ship as stable cells while still marked *preview* upstream: that label chiefly gates `-latest` alias eligibility, and no workflow here uses a floating alias. Test axes and the runners other jobs use are separate sets (`TEST_RUNNERS_*` and `NON_MATRIX_RUNNERS` in `matrix_axes.py`), and `KNOWN_RUNNERS` is their union, so a job on `ubuntu-slim` still passes `lint-repo` while a typo fails. Release binaries stay on GA images: a preview image's capacity is unbalanced for its first weeks, and queue time would delay a publish.
 
 ## Agent conventions
 

docs/test-matrix.md

@@ -535,20 +535,6 @@ pypi-package-history = [ "gha-utils", "repokit" ]
 # marketplace at .claude-plugin/marketplace.json installs the plugin straight
 # from this asset, so a release missing it breaks `/plugin install`.
 release-assets = [ "repomatic-claude-plugin.zip" ]
-# Test-drive GitHub's preview Ubuntu 26.04 images on this repository only: the
-# shipped defaults in repomatic/matrix_axes.py stay on GA images, so no
-# downstream repo inherits a preview. Both cells run continue-on-error, so a
-# preview breaking never reddens main, and neither joins the PR matrix.
-# `ubuntu-26.04-arm` against `ubuntu-24.04-arm` is the comparison worth
-# reading: same architecture, same full image, only the base version differs.
-# `ubuntu-26.04` has no matching control, the matrix's x86 Linux runner being
-# the lean `ubuntu-slim`, so it measures compatibility rather than speed.
-# See docs/test-matrix.md § Preview images under evaluation.
-test-matrix.unstable = [
-  { os = "ubuntu-26.04" },
-  { os = "ubuntu-26.04-arm" },
-]
-test-matrix.variations.os = [ "ubuntu-26.04", "ubuntu-26.04-arm" ]
 
 [tool.actionlint]
 # actionlint reads no pyproject section of its own: repomatic serializes this one

pyproject.toml

@@ -41,6 +41,7 @@
 from .github.matrix import stale_axis_values
 from .github.token import check_all_pat_permissions
 from .matrix_axes import (
+    NON_MATRIX_RUNNERS,
     TEST_RUNNERS_FULL,
     TEST_RUNNERS_PR,
     UNSTABLE_PYTHON_VERSIONS,
@@ -104,12 +105,22 @@
 major series, and `Implementation :: CPython` the interpreter.
 """
 
-KNOWN_RUNNERS = frozenset(TEST_RUNNERS_FULL) | frozenset(TEST_RUNNERS_PR)
-"""Every runner image the test matrix axes draw from.
+KNOWN_RUNNERS = (
+    frozenset(TEST_RUNNERS_FULL)
+    | frozenset(TEST_RUNNERS_PR)
+    | frozenset(NON_MATRIX_RUNNERS)
+)
+"""Every runner image this project has deliberately chosen.
 
 The closest thing to a curated list of images a project should be running on,
 and the one place carrying measured guidance on their relative speed and cost.
 A job naming something outside it has been picked without that guidance.
+
+The union of the test axes and {data}`~repomatic.matrix_axes.NON_MATRIX_RUNNERS`,
+which stopped being the same set when the test matrix moved to Ubuntu 26.04 and
+the light mechanical jobs stayed on the lean `ubuntu-slim`. Both are chosen
+deliberately, so both belong here; only an image nobody weighed should be
+flagged.
 """
 
 TEMPLATE_FILE_ARG_RE = re.compile(r"--template-file[=\s]+(?P<path>\S+)")

repomatic/lint_repo.py

@@ -29,8 +29,8 @@
     from typing import Final
 
 TEST_RUNNERS_FULL = (
-    "ubuntu-24.04-arm",
-    "ubuntu-slim",
+    "ubuntu-26.04-arm",
+    "ubuntu-26.04",
     "macos-26",
     "macos-26-intel",
     "windows-11-arm",
@@ -41,38 +41,77 @@
 Two variants per platform (one per architecture). See
 [available images](https://github.com/actions/runner-images#available-images).
 
+```{note} Preview images are adopted on measurement, not on GitHub's label
+GitHub still marks the Ubuntu 26.04 pair *preview*, which gates their
+eligibility to sit behind the `-latest` aliases. This project never uses those
+aliases (a floating alias re-points with no commit to review, which
+{func}`~repomatic.lint_repo.check_runner_images` rejects outright), so that
+distinction does not reach it. An image is treated as stable here once it has
+been validated against this suite, not once a vendor relabels it. Measured over
+consecutive runs before the swap, `ubuntu-26.04-arm` beat `ubuntu-24.04-arm` by
+16% on Python 3.10 and 28% on 3.14, tied on 3.15, and failed nothing.
+
+The residual risk is capacity rather than correctness: GitHub warns a preview
+image's capacity "will be balanced only throughout the next weeks", so queue
+time may be worse than the runtimes above suggest. Release binaries are built
+on GA images for that reason, see {data}`~repomatic.binary.NUITKA_BUILD_TARGETS`.
+```
+
 ```{note} Architecture speed is not uniform across platforms
 When reducing to one runner per OS, choose by measured speed, not architecture
 (see {doc}`/test-matrix`). Tendencies from `repomatic`'s own full test suite:
-ARM Linux (`ubuntu-24.04-arm`) runs two to three times as fast as the lean x86
-`ubuntu-slim`, the slowest tier overall;
-Apple-silicon `macos-26` beats `macos-26-intel` by ~2x; the two Windows images
-tie on compute (`windows-2025` is the PR pick). Per-job wall-clock folds in
-setup and upload, so isolate the test steps before blaming the image. These
-figures drift as images are re-provisioned, so re-confirm against your own job
-timings.
+ARM Linux runs two to three times as fast as the lean x86 `ubuntu-slim` that
+preceded `ubuntu-26.04` on this axis; Apple-silicon `macos-26` beats
+`macos-26-intel` by ~2x; the two Windows images tie on compute (`windows-2025`
+is the PR pick). Per-job wall-clock folds in setup and upload, so isolate the
+test steps before blaming the image. These figures drift as images are
+re-provisioned, so re-confirm against your own job timings.
 ```
 """
 
-TEST_RUNNERS_PR = (
+NON_MATRIX_RUNNERS = (
+    "ubuntu-slim",
+    "ubuntu-24.04",
     "ubuntu-24.04-arm",
+)
+"""Images used by jobs that are not test-matrix cells.
+
+The test axes above answer "where is the suite exercised". These answer "what
+else may a job legitimately run on", and the two stopped being the same set when
+the matrix moved to Ubuntu 26.04 while the rest of the fleet did not:
+
+- `ubuntu-slim` is the lean default for the light mechanical jobs (linters and
+  formatters), which are setup-bound rather than compute-bound, so the faster
+  image buys them nothing. GitHub rebases this image onto a newer Ubuntu on its
+  own schedule, which is why it carries no version in its name.
+- `ubuntu-24.04` and `ubuntu-24.04-arm` host the Linux Nuitka builds and the
+  one compute-heavy light job (`Format Markdown`, which needs `shfmt` and so
+  cannot use the lean image).
+
+{data}`~repomatic.lint_repo.KNOWN_RUNNERS` is the union of this and the test
+axes, so a job naming one of these is not flagged as untracked while a genuine
+typo still is.
+"""
+
+TEST_RUNNERS_PR = (
+    "ubuntu-26.04-arm",
     "macos-26",
     "windows-2025",
 )
 """Reduced runner set for pull request test matrices.
 
-One runner per platform: ARM Linux (`ubuntu-24.04-arm`) and Apple-silicon macOS
+One runner per platform: ARM Linux (`ubuntu-26.04-arm`) and Apple-silicon macOS
 (`macos-26`) are the fastest of their platform on the test workload, plus x86
 Windows (`windows-2025`, where the two Windows images tie on compute). x86 Linux
 stays covered by the full matrix ({data}`TEST_RUNNERS_FULL`).
 
 ```{note} Why ARM Linux for the PR slot
 The suite runs `pytest --numprocesses=auto`, so it scales with cores and favors
-ARM: `ubuntu-24.04-arm` runs it two to three times faster than the lean
-`ubuntu-slim`, for quicker PR feedback. That ratio is the heavy test suite's,
-not a portable property: setup-bound light jobs (which run prebuilt
-single-threaded binaries) barely move between runners, so they keep the lean
-`ubuntu-slim` default. See {doc}`/test-matrix` for the measurements.
+ARM, by two to three times over the lean x86 image, for quicker PR feedback.
+That ratio is the heavy test suite's, not a portable property: setup-bound light
+jobs (which run prebuilt single-threaded binaries) barely move between runners,
+so they keep the lean `ubuntu-slim` default ({data}`NON_MATRIX_RUNNERS`). See
+{doc}`/test-matrix` for the measurements.
 ```
 """
 
@@ -109,7 +148,7 @@
 {data}`SINGLE_RUNNER_PYTHON_VERSIONS`, which are released and run stable.
 """
 
-SINGLE_RUNNER_PYTHON_VERSIONS: Final[dict[str, str]] = {"3.14t": "ubuntu-24.04-arm"}
+SINGLE_RUNNER_PYTHON_VERSIONS: Final[dict[str, str]] = {"3.14t": "ubuntu-26.04-arm"}
 """Released Python build flavors smoke-tested on a single runner, mapped to it.
 
 A free-threaded build (the `t` suffix, made officially supported in 3.14 by
@@ -118,7 +157,7 @@
 full cross-platform spread ({data}`TEST_PYTHON_FULL`), so the library logic is
 covered everywhere; the flavor only needs one runner to catch a
 free-threading-specific break. These run *stable* (expected to pass), unlike the
-unreleased {data}`UNSTABLE_PYTHON_VERSIONS`. The runner is `ubuntu-24.04-arm`,
+unreleased {data}`UNSTABLE_PYTHON_VERSIONS`. The runner is `ubuntu-26.04-arm`,
 the default single-runner pick: the fastest measured on compute-bound parallel
 work and the cheapest tier, and free-threading targets server workloads where
 Linux/ARM is the norm (see {doc}`/test-matrix`).

repomatic/matrix_axes.py

@@ -1165,7 +1165,9 @@ def test_pypi_trusted_publisher_repository_mismatch():
     ("runner", "stale"),
     [
         pytest.param("macos-15-intel", True, id="renamed-runner"),
-        pytest.param("ubuntu-slim", False, id="live-runner"),
+        # Must be a value the test axes actually carry: `ubuntu-slim` is a known
+        # runner but no longer a matrix cell, so excluding it would be stale.
+        pytest.param("ubuntu-26.04-arm", False, id="live-runner"),
     ],
 )
 def test_check_test_matrix_excludes(tmp_path, monkeypatch, runner, stale):
@@ -1283,7 +1285,11 @@ def test_python_matrix_built_at_runtime_is_skipped(tmp_path, monkeypatch):
     ("runner", "expect_fail", "needle"),
     [
         pytest.param("ubuntu-latest", True, "repoints", id="floating-alias"),
-        pytest.param("ubuntu-24.04", True, "not one of the images", id="unknown-image"),
+        # A real GitHub image, but one this project runs nothing on: neither a
+        # test axis nor a NON_MATRIX_RUNNERS entry.
+        pytest.param("ubuntu-22.04", True, "not one of the images", id="unknown-image"),
+        # Known without being a test axis, which is the case NON_MATRIX_RUNNERS
+        # exists to cover: every light mechanical job runs here.
         pytest.param("ubuntu-slim", False, None, id="known-image"),
         pytest.param("${{ matrix.os }}", False, None, id="expression"),
     ],

tests/test_lint_repo.py

@@ -797,19 +797,14 @@ def _tracked_inventory(
             },
         ]),
     },
-    # The two ubuntu-26.04 runners and their unstable includes are this
-    # repository's own `[tool.repomatic] test-matrix` probe of GitHub's preview
-    # images, not a shipped default. Drop them here when the probe ends.
     "test_matrix": {
         "os": [
-            "ubuntu-24.04-arm",
-            "ubuntu-slim",
+            "ubuntu-26.04-arm",
+            "ubuntu-26.04",
             "macos-26",
             "macos-26-intel",
             "windows-11-arm",
             "windows-2025",
-            "ubuntu-26.04",
-            "ubuntu-26.04-arm",
         ],
         "python-version": [
             "3.10",
@@ -819,17 +814,15 @@ def _tracked_inventory(
         "include": [
             {"state": "stable"},
             {"state": "unstable", "python-version": "3.15"},
-            {"os": "ubuntu-24.04-arm", "python-version": "3.14t", "state": "stable"},
-            {"os": "ubuntu-26.04", "state": "unstable"},
-            {"os": "ubuntu-26.04-arm", "state": "unstable"},
+            {"os": "ubuntu-26.04-arm", "python-version": "3.14t", "state": "stable"},
         ],
         "exclude": [
             {"os": "windows-11-arm", "python-version": "3.10"},
         ],
     },
     "test_matrix_pr": {
         "os": [
-            "ubuntu-24.04-arm",
+            "ubuntu-26.04-arm",
             "macos-26",
             "windows-2025",
         ],
@@ -1089,7 +1082,7 @@ def test_show_test_matrix_renders_full_grid():
     result = CliRunner().invoke(repomatic, ["show-test-matrix"])
     assert result.exit_code == 0, result.output
     assert "Python" in result.output
-    assert "ubuntu-24.04-arm" in result.output
+    assert "ubuntu-26.04-arm" in result.output
     # The matrix always carries stable jobs, decorated with an emoji by default.
     assert "✅ stable" in result.output
 
@@ -1534,7 +1527,7 @@ def test_full_include_flattens_matrix(tmp_path, monkeypatch):
 [tool.repomatic]
 test-matrix.include = [{ "click-version" = "released" }]
 test-matrix.full-include = [
-  { "os" = "ubuntu-24.04-arm", "python-version" = "3.10", "click-version" = "8.3.1" },
+  { "os" = "ubuntu-26.04-arm", "python-version" = "3.10", "click-version" = "8.3.1" },
 ]
 """,
     )
@@ -1546,13 +1539,13 @@ def test_full_include_flattens_matrix(tmp_path, monkeypatch):
     rows = emitted["include"]
 
     shipped = {
-        "os": "ubuntu-24.04-arm",
+        "os": "ubuntu-26.04-arm",
         "python-version": "3.10",
         "click-version": "released",
         "state": "stable",
     }
     pinned = {
-        "os": "ubuntu-24.04-arm",
+        "os": "ubuntu-26.04-arm",
         "python-version": "3.10",
         "click-version": "8.3.1",
         "state": "stable",
@@ -1586,7 +1579,7 @@ def test_full_include_backfills_probe_defaults(tmp_path, monkeypatch):
   { "cloup-version" = "released" },
 ]
 test-matrix.full-include = [
-  { "os" = "ubuntu-24.04-arm", "python-version" = "3.14", "click-version" = "main" },
+  { "os" = "ubuntu-26.04-arm", "python-version" = "3.14", "click-version" = "main" },
 ]
 """,
     )
@@ -1596,7 +1589,7 @@ def test_full_include_backfills_probe_defaults(tmp_path, monkeypatch):
 
     probe = next(r for r in rows if r["python-version"] == "3.14t")
     assert probe == {
-        "os": "ubuntu-24.04-arm",
+        "os": "ubuntu-26.04-arm",
         "python-version": "3.14t",
         "click-version": "released",
         "cloup-version": "released",
@@ -1843,19 +1836,19 @@ def test_test_matrix_config_replace(tmp_path, monkeypatch):
 version = "1.0.0"
 
 [tool.repomatic.test-matrix.replace]
-os = { "ubuntu-24.04-arm" = "ubuntu-24.04" }
+os = { "ubuntu-26.04-arm" = "ubuntu-24.04" }
 """
     metadata = metadata_from_pyproject(tmp_path, monkeypatch, pyproject_content)
 
-    # Full matrix: ubuntu-24.04-arm replaced with ubuntu-24.04.
+    # Full matrix: ubuntu-26.04-arm replaced with ubuntu-24.04.
     full = metadata.test_matrix.matrix()
     assert "ubuntu-24.04" in full["os"]
-    assert "ubuntu-24.04-arm" not in full["os"]
+    assert "ubuntu-26.04-arm" not in full["os"]
 
-    # PR matrix: same replacement applied (ubuntu-24.04-arm is in both sets).
+    # PR matrix: same replacement applied (ubuntu-26.04-arm is in both sets).
     pr = metadata.test_matrix_pr.matrix()
     assert "ubuntu-24.04" in pr["os"]
-    assert "ubuntu-24.04-arm" not in pr["os"]
+    assert "ubuntu-26.04-arm" not in pr["os"]
 
 
 def test_test_matrix_config_remove(tmp_path, monkeypatch):
@@ -1966,17 +1959,18 @@ def test_stale_test_matrix_excludes(tmp_path, monkeypatch):
 
 [tool.repomatic.test-matrix]
 exclude = [
-    {os = "ubuntu-slim"},
+    {os = "ubuntu-26.04"},
     {os = "macos-15-intel"},
     {python-version = "9.99"},
 ]
 """
     metadata = metadata_from_pyproject(tmp_path, monkeypatch, pyproject_content)
 
-    # ubuntu-slim is a live runner, so its exclude is honored. The renamed
+    # ubuntu-26.04 is a live axis value, so its exclude is honored. The renamed
     # macos-15-intel and the bogus Python version match no axis and are flagged.
+    # `ubuntu-slim` would fail here too: a known runner, but no longer a cell.
     stale = metadata.stale_test_matrix_excludes
-    assert {"os": "ubuntu-slim"} not in stale
+    assert {"os": "ubuntu-26.04"} not in stale
     assert {"os": "macos-15-intel"} in stale
     assert {"python-version": "9.99"} in stale