Skip to content

Move container storage to the large runner disk#23

Merged
Danathar merged 1 commit into
mainfrom
ci-disk-space
Jul 2, 2026
Merged

Move container storage to the large runner disk#23
Danathar merged 1 commit into
mainfrom
ci-disk-space

Conversation

@Danathar

@Danathar Danathar commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Fixes the disk-exhaustion failure that broke the main-branch build after #22 merged (run 28593366592). The failing step is Rechunk Image with Chunkah, which is unrelated to #22's contents — the same code passed on the #22 PR build; this is an environmental disk-space failure that happens at the margin.

Root cause

After the existing "Maximize build space" step, the hosted runner has:

  • / — 47G free (holds podman's rootless storage under ~/.local/share/containers and /var/tmp)
  • /mnt — 66G free, nearly unused

The Chunkah rechunk keeps two copies of the root filesystem in podman storage at once (the built image plus the re-layered output) and podman load unpacks an OCI archive under /var/tmp. Both live on /, and together they exceeded 47G:

error deleting layer ...: no space left on device
unpacking failed (... write /usr/share/rpm/rpmdb.sqlite: no space left on device)

Fix

Relocate podman's rootless storage onto /mnt (66G free) via a symlink, before the first container operation. The image-copy temp dir is deliberately left on / — now freed by moving storage off it — so the two large consumers land on separate disks instead of competing for one.

This changes only where container storage lives on the runner; it does not touch image contents or build inputs. The PR build runs Build + Rechunk, so it exercises the fix end-to-end before merge.

🤖 Generated with Claude Code

The Chunkah rechunk step keeps two copies of the root filesystem in
podman's storage at once (the built image plus the re-layered output),
which exhausted / on the hosted runner and failed the build with
"no space left on device". Relocate podman's rootless storage onto /mnt,
which has ~66G free versus / after cleanup, leaving the image-copy temp
dir on the now-freed / so the two large consumers use separate disks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Danathar
Danathar merged commit 9b2c70b into main Jul 2, 2026
1 check passed
@Danathar
Danathar deleted the ci-disk-space branch July 2, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant