Skip to content

Commit 0937d0f

Browse files
air-31air-31
committed
Add cloud image registry implementation
1 parent 9334ec8 commit 0937d0f

18 files changed

Lines changed: 1175 additions & 39 deletions

File tree

‎.github/workflows/build-matrix.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,13 @@
1313
"build-args": "COMPONENT=tenant-operator",
1414
"harbor-project": "crownlabs-core"
1515
},
16+
{
17+
"component": "cloudimg-registry",
18+
"context": "./operators",
19+
"dockerfile": "./operators/build/golang-common/Dockerfile",
20+
"build-args": "COMPONENT=cloudimg-registry",
21+
"harbor-project": "crownlabs-core"
22+
},
1623
{
1724
"component": "bastion-operator",
1825
"context": "./operators",

‎deploy/crownlabs/Chart.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ dependencies:
6565
repository: file://../../operators/deploy/instmetrics
6666
condition: instmetrics.enabled
6767

68+
- name: cloudimg-registry
69+
version: "0.1.0"
70+
repository: file://../../operators/deploy/cloudimg-registry
71+
condition: cloudimg-registry.enabled
72+
6873
- name: policies
6974
version: "0.1.0"
7075
repository: file://../../policies

‎deploy/crownlabs/values.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,17 @@ instmetrics:
156156
updatePeriod: 4s
157157
grpcPort: 9090
158158

159+
cloudimg-registry:
160+
replicaCount: 1
161+
configurations:
162+
volume:
163+
size: "100Gi"
164+
accessMode: "ReadWriteMany"
165+
storageClass: "rook-cephfs-primary"
166+
image:
167+
repository: crownlabs/cloudimg-registry
168+
pullPolicy: IfNotPresent
169+
159170
policies:
160171
ingressHostnamePattern: s??????.sandbox.crownlabs.polito.it
161172
namespaceSelector:

‎infrastructure/cloudimage-registry/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Cloud Image Registry
2+
3+
## Table of Contents
4+
- [What is it?](#what-is-it)
5+
- [Why do we need it?](#why-do-we-need-it)
6+
- [How to use it](#how-to-use-it)
7+
8+
---
9+
10+
## What is it?
11+
12+
**Cloud Image Registry** is an image storage solution designed to store and serve VM images within a Kubernetes cluster. It consists of:
13+
- A custom **Golang server** running as a Kubernetes **Deployment**.
14+
- A **Persistent Volume Claim (PVC)** that acts as the storage backend for the images.
15+
16+
The server provides a set of HTTP endpoints to **upload, download, and manage images and their metadata**.
17+
18+
---
19+
20+
## Why do we need it?
21+
22+
There is already a private container registry, **[Harbor](../docker-registry/README.md)**, present in our setup to store VM images. The goal of introducing it was to reduce external internet traffic and improve user experience when deploying VMs.
23+
24+
However, it was discovered later that **storing VM images in a PVC inside the cluster significantly accelerates VM instantiation**. Therefore, we developed this **Cloud Image Registry** to:
25+
1. **Enhance VM startup times** by providing an internal, fast-access storage for VM images.
26+
2. **Retain image metadata**, allowing future extensions and features that depend on additional image details.
27+
28+
---
29+
30+
## How to use it
31+
32+
### **Deployment Setup**
33+
The registry currently runs inside the Kubernetes cluster and is **accessible only via an internal network**. The setup consists of:
34+
- A **Deployment** running the custom Golang server.
35+
- A **ClusterIP Service** for internal access.
36+
- A **Persistent Volume Claim (PVC)** to store image files.
37+
38+
**Note:** Future updates may include exposing the service externally via an **Ingress** resource.
39+
40+
### **Configuration Options**
41+
Some server options can be configured through **command-line flags** in the deployment [manifest](../../operators/deploy/cloudimg-registry/templates/deployment.yaml):
42+
- `--listener-addr` → Server listen address (default: `localhost:8080`)
43+
- `--data-root` → Root directory of the registry inside PVC
44+
- `--read-header-timeout-secs` → Timeout for reading headers in HTTP requests
45+
46+
### **API Endpoints**
47+
All interactions happen via HTTP requests. Below is a table summarizing the available endpoints:
48+
49+
| Method | Endpoint | Description |
50+
|--------|----------------------------------|-----------------------------|
51+
| **GET** | `/reg` | Lists all existing repositories. |
52+
| **GET** | `/reg/{repo}` | Lists all images inside a repository. |
53+
| **GET** | `/reg/{repo}/{image}` | Lists all available tags for an image. |
54+
| **GET** | `/reg/{repo}/{image}/{tag}` | Serves the image file for an image tag. |
55+
| **GET** | `/reg/{repo}/{image}/{tag}/meta` | Serves the metadata JSON file for an image tag. |
56+
| **POST** | `/reg/{repo}/{image}/{tag}` | Uploads an image with its metadata. **Request body:** `meta` (metadata JSON), `img` (image file). |
57+
| **DELETE** | `/reg/{repo}/{image}/{tag}` | Deletes an image and its metadata. If it's the last tag/image, the image/repo directory is removed. |
58+
59+
**Note:** `reg` in the endpoints above is a default registry prefix which can also be configured using `--repo-prefix` command-line flag.
60+
61+
**Example Base URL:**
62+
For a server running at `192.168.0.1:8080`, with the default registry prefix (`reg`), the following request would be valid:
63+
64+
- **List all repositories:**
65+
```sh
66+
curl -X GET http://192.168.0.1:8080/reg

‎operators/cmd/cloudimg-registry/main.go‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Copyright 2020-2025 Politecnico di Torino
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
// Package main contains the entrypoint for the cloud image registry.
16+
package main
17+
18+
import (
19+
"flag"
20+
21+
"k8s.io/klog/v2"
22+
23+
"github.com/netgroup-polito/CrownLabs/operators/pkg/ciregistry"
24+
)
25+
26+
var (
27+
dataRoot = flag.String("data-root", "/data", "Root data path for the server")
28+
serverPrefix = flag.String("www-prefix", "/", "Starting path where the web server should run from")
29+
registryPrefix = flag.String("repo-prefix", "reg", "Subpath of data location on web")
30+
listenerAddr = flag.String("listener-addr", ":8080", "Address for the server to listen on")
31+
readHeaderTimeoutSeconds = flag.Int("read-header-timeout-secs", 2, "Number of seconds allowed to read request headers")
32+
)
33+
34+
func main() {
35+
// Initialize klog
36+
klog.InitFlags(nil)
37+
defer klog.Flush()
38+
39+
// Parse flags
40+
flag.Parse()
41+
42+
// Start the server
43+
ciregistry.Start(*dataRoot, *serverPrefix, *registryPrefix, *listenerAddr, *readHeaderTimeoutSeconds)
44+
}

‎operators/deploy/cloudimg-registry/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

‎operators/deploy/cloudimg-registry/Chart.yaml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
apiVersion: v2
2+
name: cloudimg-registry
3+
description: The CrownLabs Cloud Image Registry
4+
5+
# A chart can be either an 'application' or a 'library' chart.
6+
#
7+
# Application charts are a collection of templates that can be packaged into versioned archives
8+
# to be deployed.
9+
#
10+
# Library charts provide useful utilities or functions for the chart developer. They're included as
11+
# a dependency of application charts to inject those utilities and functions into the rendering
12+
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
13+
type: application
14+
15+
# This is the chart version. This version number should be incremented each time you make changes
16+
# to the chart and its templates, including the app version.
17+
# Versions are expected to follow Semantic Versioning (https://semver.org/)
18+
version: 0.1.0
19+
20+
icon: https://crownlabs.polito.it/images/logo.svg

‎operators/deploy/cloudimg-registry/templates/_helpers.tpl‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
{{/* vim: set filetype=mustache: */}}
2+
{{/*
3+
Expand the name of the chart.
4+
*/}}
5+
{{- define "cloudimg-registry.name" -}}
6+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
7+
{{- end }}
8+
9+
{{/*
10+
Create a default fully qualified app name.
11+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
12+
If the release name contains the chart name, it will be used as a full name.
13+
*/}}
14+
{{- define "cloudimg-registry.fullname" -}}
15+
{{- if .Values.fullnameOverride }}
16+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
17+
{{- else }}
18+
{{- $name := default .Chart.Name .Values.nameOverride }}
19+
{{- if contains $name .Release.Name }}
20+
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
21+
{{- else }}
22+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
23+
{{- end }}
24+
{{- end }}
25+
{{- end }}
26+
27+
{{/*
28+
The version of the application to be deployed
29+
*/}}
30+
{{- define "cloudimg-registry.version" -}}
31+
{{- if .Values.global }}
32+
{{- .Values.image.tag | default .Values.global.version | default .Chart.AppVersion }}
33+
{{- else }}
34+
{{- .Values.image.tag | default .Chart.AppVersion }}
35+
{{- end }}
36+
{{- end }}
37+
38+
{{/*
39+
Create chart name and version as used by the chart label.
40+
*/}}
41+
{{- define "cloudimg-registry.chart" -}}
42+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
43+
{{- end }}
44+
45+
{{/*
46+
Common labels
47+
*/}}
48+
{{- define "cloudimg-registry.labels" -}}
49+
helm.sh/chart: {{ include "cloudimg-registry.chart" . }}
50+
{{ include "cloudimg-registry.selectorLabels" . }}
51+
app.kubernetes.io/version: {{ include "cloudimg-registry.version" . | quote }}
52+
app.kubernetes.io/managed-by: {{ .Release.Service }}
53+
{{- end }}
54+
55+
{{/*
56+
Selector labels
57+
*/}}
58+
{{- define "cloudimg-registry.selectorLabels" -}}
59+
app.kubernetes.io/name: {{ include "cloudimg-registry.name" . }}
60+
app.kubernetes.io/instance: {{ .Release.Name }}
61+
{{- end }}

‎operators/deploy/cloudimg-registry/templates/deployment.yaml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: {{ include "cloudimg-registry.fullname" . }}
5+
labels:
6+
{{ include "cloudimg-registry.labels" . | nindent 4 }}
7+
{{- with .Values.deploymentAnnotations }}
8+
annotations:
9+
{{- toYaml . | nindent 4 }}
10+
{{- end }}
11+
spec:
12+
replicas: {{ .Values.replicaCount }}
13+
selector:
14+
matchLabels:
15+
{{ include "cloudimg-registry.selectorLabels" . | nindent 6 }}
16+
template:
17+
metadata:
18+
{{- with .Values.podAnnotations }}
19+
annotations:
20+
{{- toYaml . | nindent 8 }}
21+
{{- end }}
22+
labels:
23+
{{- include "cloudimg-registry.selectorLabels" . | nindent 8 }}
24+
spec:
25+
{{- with .Values.imagePullSecrets }}
26+
imagePullSecrets:
27+
{{- toYaml . | nindent 8 }}
28+
{{- end }}
29+
initContainers:
30+
- name: fix-permissions
31+
image: busybox:1.36.1
32+
command: ["sh", "-c", "chown -R {{ .Values.podSecurityContext.fsGroup }}:{{ .Values.podSecurityContext.fsGroup }} {{ .Values.configurations.dataRoot }}"]
33+
resources:
34+
{{- toYaml .Values.resources | nindent 10 }}
35+
volumeMounts:
36+
- name: "{{ include "cloudimg-registry.fullname" . }}-storage"
37+
mountPath: {{ .Values.configurations.dataRoot }}
38+
containers:
39+
- name: {{ .Chart.Name }}
40+
image: "{{ .Values.image.repository }}:{{ include "cloudimg-registry.version" . }}"
41+
imagePullPolicy: {{ .Values.image.pullPolicy }}
42+
args:
43+
- "--data-root={{ .Values.configurations.dataRoot }}"
44+
- "--read-header-timeout-secs={{ .Values.configurations.readHeaderTimeoutSeconds }}"
45+
- "--listener-addr=:8080"
46+
ports:
47+
- name: http
48+
containerPort: 8080
49+
protocol: TCP
50+
readinessProbe:
51+
httpGet:
52+
path: /healthz
53+
port: http
54+
initialDelaySeconds: 3
55+
periodSeconds: 3
56+
resources:
57+
{{- toYaml .Values.resources | nindent 12 }}
58+
volumeMounts:
59+
- name: "{{ include "cloudimg-registry.fullname" . }}-storage"
60+
mountPath: {{ .Values.configurations.dataRoot }}
61+
securityContext:
62+
{{- toYaml .Values.securityContext | nindent 12 }}
63+
securityContext:
64+
{{- toYaml .Values.podSecurityContext | nindent 8 }}
65+
volumes:
66+
- name: "{{ include "cloudimg-registry.fullname" . }}-storage"
67+
persistentVolumeClaim:
68+
claimName: "{{ include "cloudimg-registry.fullname" . }}-pvc"

‎operators/deploy/cloudimg-registry/templates/pvc.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: v1
2+
kind: PersistentVolumeClaim
3+
metadata:
4+
name: "{{ include "cloudimg-registry.fullname" . }}-pvc"
5+
labels:
6+
{{ include "cloudimg-registry.labels" . | nindent 4 }}
7+
spec:
8+
accessModes:
9+
- {{ .Values.configurations.volume.accessMode }}
10+
resources:
11+
requests:
12+
storage: {{ .Values.configurations.volume.size }}
13+
storageClassName: {{ .Values.configurations.volume.storageClass }}

0 commit comments

Comments
 (0)