feat(build): cosign-verify + digest-pin pre-bundled control-plane images (ADR-16 P5)#25
Merged
Merged
Conversation
…ges (ADR-16 P5)
Harden the air-gap image bundle (kairos#4205 P5): the pre-bundled control-plane
images land in the immutable OS and are ctr-imported at boot, bypassing any
runtime CRI image policy, so provenance is established at build time.
The image-bundler now, per image kubeadm requires:
1. resolves the floating tag to an immutable digest once (crane digest),
2. cosign-verifies that digest against the Kubernetes krel keyless identity
(krel-trust@k8s-releng-prod.iam.gserviceaccount.com / accounts.google.com),
3. pulls the SAME digest (TOCTOU-safe: verify and pull bound to one digest).
cosign and crane are version-pinned and checksum-verified at install.
registry.k8s.io signature coverage is empirically incomplete: kube-apiserver,
kube-controller-manager and kube-proxy carry no krel signature (confirmed by
identical digest + absent .sig/referrers at the canonical us-central1 backend;
k8s#129199), while kube-scheduler/coredns/pause/etcd do. So verification is
best-effort with a hard floor:
- signature verifies -> recorded verified=true.
- cosign "no signatures found" -> recorded verified=false / no-upstream-signature
(digest-pinned only; still content-addressed).
- any other cosign failure -> hard fail (present-but-invalid sig or error).
- floor (build-blocking): pause+etcd+coredns AND >=1 kube-* MUST verify, each a
positive pass (a timeout cannot satisfy the floor).
A baked /opt/provider-kubernetes/images/images.lock records ref->digest plus the
per-image verified flag/reason; the release pipeline converts it to a CycloneDX
SBOM (build/gen-bundled-sbom.py) and attests it, so each released image attests
exactly which control-plane images it baked and at what trust level (ADR-15).
.gitignore/.dockerignore un-ignore the two build/ helper scripts (vmtest
artifacts stay ignored). security-architect gate: PASS (B1a/B1b/B1c with R1/R2,
B2-B7 verified in code); residual risk recorded (R-resid-1/-2).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: William Rizzo <william.rizzo@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Harden the air-gap image bundle (ADR-16 P5, the deferred security-gated slice of
kairos-io/kairos#4205). The pre-bundled control-plane images land in the immutable OS
and are
ctr imported at boot, bypassing any runtime CRI image policy — so provenanceis established at build time via cosign signature verification + digest pinning.
How
The
image-bundlerstage, for each image kubeadm requires:crane digest),(
krel-trust@k8s-releng-prod.iam.gserviceaccount.com/https://accounts.google.com),crane pull ref@digest) — verify and pull are bound toone digest (TOCTOU-safe).
cosignandcraneare version-pinned and checksum-verified at install.Best-effort verification with a hard floor
registry.k8s.io signature coverage is empirically incomplete:
kube-apiserver,kube-controller-manager, andkube-proxycarry no krel signature (confirmed byidentical digest + absent
.sig/referrers at the canonicalus-central1-docker.pkg.devbackend — a known upstream condition, k8s#129199), while
kube-scheduler,coredns,pause,etcddo verify. So:verified: true.no signatures found→ recordedverified: false / no-upstream-signature(digest-pinned only; still content-addressed).error) → hard fail.
pause+etcd+corednsAND ≥1kube-*component MUSTverify — each a positive pass (a timeout can't satisfy the floor).
Auditability
A baked
/opt/provider-kubernetes/images/images.lockrecordsref → digestplus theper-image
verifiedflag/reason. The release pipeline converts it to a CycloneDX SBOM(
build/gen-bundled-sbom.py) and attests it, so each released image attests exactlywhich control-plane images it baked and at what trust level (ADR-15).
Validation
make imagegreen; the bundler stage exercised cosign against registry.k8s.iofor real → 7 images, 4 signature-verified, 3 recorded
no-upstream-signature.images.lock+ 7 tarballs confirmed in the final image;sh -n+ shellcheck clean;release YAML valid; Go gates green.
Security review
security-architect gate: PASS — B1a/B1b/B1c (with R1: precise no-signature match,
R2: positive-pass floor), B2–B7 all verified in code. Residual risk recorded
(R-resid-1: the kube-* trio is digest-pinned-only and upstream-unsigned; B1a
auto-tightens to require their signatures if/when upstream starts signing them.
R-resid-2: backlog CI canary to detect that change). Two should-fix nits actioned
inline (anchored floor matches; documented closed-enum
verifyReason).