CI: cross-run Docker layer cache via BuildKit (deferred)

[hi-liang]

Summary

CI has no cross-run Docker layer cache, so every run rebuilds the same image layers from scratch — the foundation apt-get installs, the solver pip install pwntools, and especially the disks challenge's libguestfs install. Adopting a BuildKit-backed builder with an exportable cache (e.g. GitHub Actions type=gha) would make unchanged layers cache hits across runs.

Not blocking / do not start yet: deferred until the in-flight Docker/moby work lands, to avoid churning the build path twice.

Why the current builder can't do this

executeBuild calls the moby SDK's classic builder directly (client.ImageBuild → POST /build) in cmgr/docker.go. The classic endpoint cannot export or import a cross-run layer cache — which is exactly the capability needed. Setting Version: BuilderBuildKit on the existing options is not sufficient: BuildKit via the moby client requires a live gRPC session (fsutil context sync, auth providers) and still cannot do registry/gha cache export without the full buildkit client. The deps confirm the gap: cmgr pulls in github.com/moby/moby/client and docker/cli/opts, but not moby/buildkit or docker/buildx.

So this is an integration, not a flag flip.

Options

Route	Lift	Gets us	Cost / caveat
A. moby/buildkit client (Solve + cache exporters)	Large	Full control, gha/registry/local cache	New heavyweight dep; rewrite the build call + error parsing; manage a buildkitd/builder on CI runners
B. Shell out to docker buildx build with --cache-to/from type=gha	Medium (~a day)	First-class cache backends, minimal cmgr code	Architectural departure — cmgr currently has no docker-CLI dependency; requires buildx present; pipe the existing context tar via stdin; re-parse buildx stderr progress instead of the JSON stream
C. Existing CacheFrom + CMGR_REGISTRY freeze path (cmgr/docker.go, ImagePull → CacheFrom)	Small	Cross-run cache with the classic builder, no new deps	Caches per-challenge frozen bases, not the shared foundation; needs a registry + a cmgr freeze step wired into CI

Expected gain

Within a single run, Docker's local cache already builds each foundation once and reuses it, so the cross-run win is: skip rebuilding the foundation apt layers, the pwntools solver install, and especially disks' libguestfs (~2-4 min on its own) on any run where Dockerfiles/context are unchanged — i.e. every code-only PR. Roughly 3-5 min off a ~10 min run, minus gha export/import overhead (~10-30s) and full invalidation whenever a Dockerfile or build context changes.

Recommendation

Route B (buildx shell-out) is the best effort-to-reward if we want the layer cache. Suggest gating it behind an env flag (e.g. CMGR_BUILDKIT=1) so it's opt-in and the default SDK path is undisturbed, letting us measure the real CI delta before committing.

Notes / dependencies

• The shared-foundation Dockerfile standardization (identical early layers across challenge/type Dockerfiles) is the precondition for any of these to pay off; it already helps within-run today.
• The Go module/build cache was a separate, already-fixed CI issue (setup-go ran before checkout, so its go.sum-keyed cache never restored).
• Blocked on: in-flight Docker/moby work — revisit after that lands.