Skip to content

Commit ba97395

Browse files
committed
perf(haos-e2e): compress qcow2 in-format before oras push + add tag_suffix dispatch input
The publish workflow currently pushes the post-bake qcow2 raw via ORAS (``haos-test-image.qcow2:application/octet-stream``), which is ~12 GB on disk and downloads at GHCR's ~32 MB/s single-stream cap → 6m 24s on every cache-miss e2e run (run #283 job 77593061250). The image is dominated by sparse-zeroed space from ``qemu-img resize 32G`` plus addon Docker layers; ``qemu-img convert -c -O qcow2`` re-packs the file with the format's native zlib compression and typically shrinks it ~3–5x. The result stays a standard qcow2 (same OCI artifact-type, same consumer code path). qemu decompresses sectors lazily during VM I/O when the e2e workflow boots it — no upfront xz-style decompress step on the pull side. Workflow changes ---------------- * New ``Compress qcow2 in-format (qemu-img convert -c)`` step before the artifact upload + GHCR push. Logs ``before / after / ratio`` so the size win is visible in the workflow output. * ``workflow_dispatch`` gains a ``tag_suffix`` input (default ``latest``) so perf-iteration branches can publish to a non-``latest`` moving tag (e.g. ``:17.3-haose2eefficiency``) without disturbing the master-served image other PRs pull from. Pull-side validation -------------------- This commit only changes how the image is *published*; the e2e workflows still pull ``:HAOS_VERSION-latest`` (which is still the old uncompressed image) until commit 3 in this series flips them to the new tag. The e2e test workflows are not in this workflow file's trigger ``paths`` and are not in this PR's commit yet either, so this push does not auto-trigger an e2e run that would pull a not-yet-existing image. The next step is a manual ``gh workflow run build-haos-test-image.yml --ref perf/haos-e2e-improvements -f tag_suffix=haose2eefficiency`` to publish the compressed image at ``:17.3-haose2eefficiency``.
1 parent 8655bd8 commit ba97395

1 file changed

Lines changed: 53 additions & 3 deletions

File tree

.github/workflows/build-haos-test-image.yml

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,17 @@ name: Build HAOS Test Image (Publish to GHCR)
2121

2222
on:
2323
workflow_dispatch:
24+
inputs:
25+
tag_suffix:
26+
description: >-
27+
Moving-tag suffix (default: ``latest``). Override to publish
28+
to ``:HAOS_VERSION-<suffix>`` instead of ``:HAOS_VERSION-latest``
29+
— used by perf-iteration branches that need a non-latest tag
30+
they can point the e2e test workflows at without touching the
31+
master-served image.
32+
type: string
33+
required: false
34+
default: 'latest'
2435
push:
2536
branches: [master]
2637
paths:
@@ -146,9 +157,42 @@ jobs:
146157
if-no-files-found: warn
147158
retention-days: 14
148159

160+
- name: Compress qcow2 in-format (qemu-img convert -c)
161+
# Re-pack the qcow2 with in-format zlib compression before upload.
162+
# The HAOS image is ~12 GB on disk after the bake, almost all of
163+
# which is sparse-zeroed space from the ``qemu-img resize 32G``
164+
# in ``fetch_haos_qcow2`` plus addon Docker layers. ``-c`` is the
165+
# qcow2-native compression flag — the resulting file stays a
166+
# standard qcow2 (same media type, same consumer code path), and
167+
# qemu decompresses sectors lazily during VM I/O when the e2e
168+
# workflow boots it. No upfront decompress step on the pull side.
169+
#
170+
# Wins:
171+
# - ``oras push``/``oras pull`` wire bytes scale with the file
172+
# size; a ~3–5 GB compressed qcow2 cuts the 6-minute single-
173+
# stream GHCR pull on cache-miss runs to ~2 min.
174+
# - actions/cache also benefits: the existing zstdmt over a
175+
# compressed qcow2 still wins on the addon Docker layer bytes
176+
# that zstd hasn't already squashed.
177+
#
178+
# Cost: one extra ``qemu-img convert`` pass on the publish runner
179+
# (~1–2 min on a 12 GB sparse qcow2), which only runs on
180+
# master push / weekly cron / manual dispatch — not on PR runs.
181+
run: |
182+
orig_bytes=$(stat -c%s haos-test-image.qcow2)
183+
qemu-img convert -p -c -O qcow2 \
184+
haos-test-image.qcow2 haos-test-image.qcow2.compressed
185+
mv haos-test-image.qcow2.compressed haos-test-image.qcow2
186+
new_bytes=$(stat -c%s haos-test-image.qcow2)
187+
ratio=$(awk -v o="$orig_bytes" -v n="$new_bytes" 'BEGIN{printf "%.1f", o/n}')
188+
echo "qcow2 size before: $(numfmt --to=iec "$orig_bytes")"
189+
echo "qcow2 size after : $(numfmt --to=iec "$new_bytes")"
190+
echo "compression ratio: ${ratio}x"
191+
149192
- name: Upload built image as workflow artifact
150193
# Always upload — even on PRs that don't push to GHCR — so reviewers
151-
# can pull the artifact and verify locally.
194+
# can pull the artifact and verify locally. (Now also smaller thanks
195+
# to the in-format compression step above.)
152196
uses: actions/upload-artifact@v7
153197
with:
154198
name: haos-test-image-${{ steps.haos_version.outputs.version }}-${{ github.sha }}
@@ -166,13 +210,19 @@ jobs:
166210
env:
167211
HAOS_VERSION: ${{ steps.haos_version.outputs.version }}
168212
SHORT_SHA: ${{ github.sha }}
213+
# workflow_dispatch lets perf-iteration branches publish to a
214+
# non-``latest`` moving tag (e.g. ``:17.3-haose2eefficiency``)
215+
# so they can point the e2e test workflows at a candidate image
216+
# without disturbing the master-served ``:HAOS_VERSION-latest``
217+
# that every other PR pulls from.
218+
TAG_SUFFIX: ${{ github.event.inputs.tag_suffix || 'latest' }}
169219
run: |
170220
short_sha="${SHORT_SHA:0:7}"
171221
versioned="${IMAGE_REPO}:${HAOS_VERSION}-${short_sha}"
172-
moving="${IMAGE_REPO}:${HAOS_VERSION}-latest"
222+
moving="${IMAGE_REPO}:${HAOS_VERSION}-${TAG_SUFFIX}"
173223
oras push "$versioned" \
174224
--artifact-type application/vnd.homeassistant-ai.haos-test-image.v1 \
175225
haos-test-image.qcow2:application/octet-stream
176-
oras tag "$versioned" "${HAOS_VERSION}-latest"
226+
oras tag "$versioned" "${HAOS_VERSION}-${TAG_SUFFIX}"
177227
echo "Pushed $versioned"
178228
echo "Pushed $moving"

0 commit comments

Comments
 (0)