Skip to content

Commit 2add850

Browse files
committed
Add blog post about publishing Kubernetes SIG images to registry.k8s.io
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
1 parent ee7fc71 commit 2add850

4 files changed

Lines changed: 412 additions & 0 deletions

File tree

Lines changed: 116 additions & 0 deletions
Loading
Lines changed: 30 additions & 0 deletions
Loading
Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
---
2+
layout: blog
3+
title: "Publishing a Kubernetes SIG's Images to registry.k8s.io"
4+
draft: true
5+
slug: publishing-images-to-registry-k8s-io
6+
author: >
7+
[Kahiro Okina](https://github.com/kahirokunn) (Craftsman Software, Inc.)
8+
---
9+
10+
For many Kubernetes SIG projects, shipping container images eventually
11+
becomes necessary, and `registry.k8s.io` is the official channel for them. I recently published a
12+
SIG project's first images there. No single step is hard, but the steps live
13+
in four repositories and have to happen in a particular order, which I mostly
14+
learned by tripping over them.
15+
16+
This post is the guide I wish I had at the start. The example project is
17+
`cluster-inventory-api` from
18+
[SIG Multicluster](https://github.com/kubernetes/community/tree/master/sig-multicluster),
19+
but nothing in the procedure is specific to that SIG.
20+
21+
## What Cluster Inventory API publishes
22+
23+
Cluster Inventory API helps applications and tools work with multiple
24+
Kubernetes clusters. It publishes the
25+
[`secretreader`](https://github.com/kubernetes-sigs/cluster-inventory-api/tree/main/plugins/secretreader/cmd/plugin)
26+
and
27+
[`kubeconfig-secretreader`](https://github.com/kubernetes-sigs/cluster-inventory-api/tree/main/plugins/kubeconfig-secretreader/cmd/plugin)
28+
access-provider plugins as OCI images. Consumers mount these images as
29+
[image volumes](https://kubernetes.io/docs/tasks/configure-pod-container/image-volumes/).
30+
31+
## My first plan: ghcr.io
32+
33+
I first tried a common GitHub release pattern: using GitHub Actions to publish
34+
images to `ghcr.io` on a tag push
35+
([cluster-inventory-api#40](https://github.com/kubernetes-sigs/cluster-inventory-api/pull/40)).
36+
The workflow succeeded, but Kubernetes GitHub organizations keep GHCR packages
37+
private, so GHCR cannot be used for public distribution.
38+
39+
{{< figure src="ghcr-path-blocked.svg" class="text-center" width="660" alt="The blocked GHCR path: a tag push triggers GitHub Actions, which pushes the image to ghcr.io, where it cannot be made public." >}}
40+
41+
As described in the
42+
[artifacts documentation](https://github.com/kubernetes/k8s.io/tree/main/artifacts#staging-buckets),
43+
official images take a different route:
44+
[Prow](https://docs.prow.k8s.io/) (the Kubernetes project's CI/CD system)
45+
picks up a tag push and runs Google Cloud Build on Kubernetes-owned
46+
infrastructure to push the image to a staging registry, and the image promoter
47+
then copies it to `registry.k8s.io`.
48+
49+
{{< figure src="official-publishing-path.svg" class="text-center" width="820" alt="The official publishing path: a tag push triggers a Prow postsubmit job, which runs Cloud Build and pushes to a staging registry. The image promoter then copies the image to registry.k8s.io." >}}
50+
51+
For this project, the images that finally shipped through that route were:
52+
53+
```none
54+
registry.k8s.io/cluster-inventory-api/secretreader:v0.1.3
55+
registry.k8s.io/cluster-inventory-api/kubeconfig-secretreader:v0.1.3
56+
```
57+
58+
## The first-time setup, step by step
59+
60+
Besides the image-owning repository, this touches three infrastructure
61+
repositories: [`kubernetes/k8s.io`](https://github.com/kubernetes/k8s.io),
62+
[`kubernetes/test-infra`](https://github.com/kubernetes/test-infra), and
63+
[`kubernetes/org`](https://github.com/kubernetes/org). The pieces depend on
64+
each other like this:
65+
66+
{{< figure src="dependency-overview.svg" alt="A dependency graph for first-time registry.k8s.io image publishing. The image-owning repository provides cloudbuild.yaml and make release-staging, and a signed release tag triggers the kubernetes/test-infra image-pushing postsubmit job. In kubernetes/k8s.io, a staging Google Group must exist before the staging registry can be created, and that registry is the job's push target. The postsubmit job pushes the staging image. That staging image, together with image promoter config in kubernetes/k8s.io and OWNERS validation through kubernetes/org membership, promotes the image to registry.k8s.io." >}}
67+
68+
### Before you start: choose registry paths, tag policy, and owners
69+
70+
Reading `registry.k8s.io/cluster-inventory-api/secretreader:v0.1.3` from the
71+
example above: `<project>` is `cluster-inventory-api`, `<image>` is
72+
`secretreader`, and `v<version>` is `v0.1.3`. For your project, decide:
73+
74+
- `<project>`, which also fixes the staging path
75+
`us-central1-docker.pkg.dev/k8s-staging-images/<project>`.
76+
- `<image>` for each image you ship.
77+
- The tag policy behind `<version>`.
78+
- Which SIG owns the project, and who reviews and approves.
79+
- Who goes in the promotion `OWNERS` file.
80+
- The staging access group name.
81+
82+
### 1. Make the image-owning repository build images
83+
84+
Set up the repository so that a tag push can build and push a staging image.
85+
You need:
86+
87+
- a `RELEASE.md` documenting the release steps,
88+
- a `cloudbuild.yaml` (the `test-infra` job in step 4 invokes this to build
89+
and push the image),
90+
- a Dockerfile and/or Make target to build the image,
91+
- a release target that pushes to the staging registry (for example
92+
`make release-staging`).
93+
94+
References:
95+
[cluster-inventory-api#53](https://github.com/kubernetes-sigs/cluster-inventory-api/pull/53)
96+
(moving to the Prow/Cloud Build approach) and
97+
[cluster-inventory-api#57](https://github.com/kubernetes-sigs/cluster-inventory-api/pull/57)
98+
(passing the staging repository explicitly to `kpromo`).
99+
100+
### 2. Add a Google Group for staging artifacts
101+
102+
Create the Google Group that will get push access to the staging registry, in
103+
your SIG's group configuration under `groups/` in `kubernetes/k8s.io`
104+
([kubernetes/k8s.io#9385](https://github.com/kubernetes/k8s.io/pull/9385)),
105+
and get approval from your SIG leads or chairs. Keep the group-name suffix
106+
within the 18-character limit
107+
([kubernetes/k8s.io#9402](https://github.com/kubernetes/k8s.io/pull/9402)).
108+
109+
### 3. Add a staging registry in kubernetes/k8s.io
110+
111+
Add one entry to the `registries` map in
112+
`infra/gcp/terraform/k8s-staging-images/registries.tf`, mapping `<project>` to
113+
the group from step 2. The module gives that group writer access and makes the
114+
repository publicly readable. Reference:
115+
[kubernetes/k8s.io#9347](https://github.com/kubernetes/k8s.io/pull/9347).
116+
117+
### 4. Add an image-pushing postsubmit job in kubernetes/test-infra
118+
119+
Add a job under `config/jobs/image-pushing/` that runs the image-owning
120+
repository's `cloudbuild.yaml` on a tag push and pushes to the staging
121+
registry. Reference:
122+
[kubernetes/test-infra#36821](https://github.com/kubernetes/test-infra/pull/36821).
123+
124+
### 5. Push a release tag to build a staging image
125+
126+
With everything above in place, push a
127+
[signed tag](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-tags)
128+
from the image-owning repository:
129+
130+
```bash
131+
git tag -s v<version>
132+
git push origin v<version>
133+
gh release create v<version> --draft --generate-notes --verify-tag
134+
```
135+
136+
Then verify the staging image:
137+
138+
```bash
139+
docker manifest inspect us-central1-docker.pkg.dev/k8s-staging-images/<project>/<image>:v<version>
140+
```
141+
142+
Tag events are not processed retroactively: tags created before the release
143+
pipeline existed will not produce a staging image.
144+
145+
### 6. Add the image promoter configuration in kubernetes/k8s.io
146+
147+
Open a `kubernetes/k8s.io` PR that adds the promotion configuration for this
148+
project
149+
([kubernetes/k8s.io#9499](https://github.com/kubernetes/k8s.io/pull/9499)):
150+
151+
- `registry.k8s.io/images/k8s-staging-<project>/OWNERS`,
152+
- `registry.k8s.io/images/k8s-staging-<project>/images.yaml` (the promotion
153+
target),
154+
- `registry.k8s.io/manifests/k8s-staging-<project>/promoter-manifest.yaml`.
155+
156+
For the first promotion, include the digest and tag entries for the staging
157+
images in `images.yaml`, and get `/lgtm` from a SIG lead. For later releases,
158+
[`kpromo`](https://github.com/kubernetes-sigs/promo-tools) generates this PR
159+
for you (see the routine release steps below). If you are curious how the
160+
promotion machinery works, see
161+
[The Invisible Rewrite: Modernizing the Kubernetes Image Promoter](https://kubernetes.io/blog/2026/03/17/image-promoter-rewrite/).
162+
163+
If anyone you plan to list in `OWNERS` is not yet a Kubernetes organization
164+
member, submit a membership request first
165+
([kubernetes/org#6385](https://github.com/kubernetes/org/pull/6385),
166+
[kubernetes/org#6386](https://github.com/kubernetes/org/pull/6386)).
167+
168+
### 7. Verify the release and publish it
169+
170+
Once the promotion PR merges, run the project's release verification and
171+
confirm the production image is available:
172+
173+
```bash
174+
docker manifest inspect registry.k8s.io/<project>/<image>:v<version>
175+
```
176+
177+
When that works, publish or update the GitHub release and announce it in the
178+
related issues and Slack channels.
179+
180+
### Where to ask for help
181+
182+
Several of these steps depend on other people: reviewers, approvers, and SIG
183+
leads. Expect to wait on reviews between steps rather than finishing in one
184+
sitting. On the [Kubernetes Slack](https://slack.k8s.io/), these channels line
185+
up with the work:
186+
187+
| Channel | Use it for |
188+
| --- | --- |
189+
| `#github-management` | Repository access, GHCR questions, and Kubernetes organization membership |
190+
| `#sig-k8s-infra` | The staging Google Group and staging registry |
191+
192+
## After the first time, it is much lighter
193+
194+
Routine releases only touch the image-owning repository and one promotion PR:
195+
196+
1. Push a signed tag, create a draft GitHub release, and confirm the
197+
postsubmit pushed the staging image, as in step 5 of the first-time setup.
198+
2. Create the promotion PR with `kpromo pr`, naming the Artifact Registry
199+
staging repository explicitly with `--staging-repo`:
200+
201+
```bash
202+
kpromo pr \
203+
--fork <your-github-username> \
204+
--project <project> \
205+
--tag v<version> \
206+
--staging-repo us-central1-docker.pkg.dev/k8s-staging-images/<project>
207+
```
208+
209+
3. Once the promotion PR is reviewed and merged, finish as in step 7 of the
210+
first-time setup.
211+
212+
## Acknowledgments
213+
214+
Thanks to [Mike Ng](https://github.com/mikeshng) and
215+
[Laura Lorenz](https://github.com/lauralorenz) for attending meetings on my
216+
behalf, connecting me with the right people, and coordinating the work across
217+
SIG Multicluster; [Jian Qiu](https://github.com/qiujian16) for reviewing the
218+
implementation;
219+
[Stephen Kitt](https://github.com/skitt) for reviewing the release process and
220+
clarifying the publishing rules; and
221+
[Arnaud M.](https://github.com/ameukam) for reviewing the `kubernetes/k8s.io`
222+
pull requests and guiding the infrastructure and promotion changes.
223+
224+
## References
225+
226+
- [cluster-inventory-api releases](https://github.com/kubernetes-sigs/cluster-inventory-api/releases)
227+
- [Using Plugin OCI Images](https://github.com/kubernetes-sigs/cluster-inventory-api/blob/main/docs/plugin-images.md)
228+
- [Image volumes](https://kubernetes.io/docs/tasks/configure-pod-container/image-volumes/)
229+
- [registry.k8s.io: faster, cheaper and Generally Available (GA)](https://kubernetes.io/blog/2022/11/28/registry-k8s-io-faster-cheaper-ga/)
230+
- [Publishing official artifact images (`kubernetes/k8s.io/artifacts`)](https://github.com/kubernetes/k8s.io/tree/main/artifacts#staging-buckets)
231+
- [`kpromo` (promo-tools)](https://github.com/kubernetes-sigs/promo-tools)
232+
- [Kubernetes Slack](https://slack.k8s.io/)

0 commit comments

Comments
 (0)