Skip to content

Commit dce6471

Browse files
yanmxaclaude
andauthored
🌱 Enable TLS for Flower addon SuperLink-SuperNode communication (#112)
* 🌱 Enable TLS for Flower addon SuperLink-SuperNode communication Add TLS support for securing gRPC communication between SuperLink (hub) and SuperNode (managed clusters). Includes a certificate generation script and Helm chart conditional TLS configuration. - Add hack/generate-certs.sh to generate CA + server cert as K8s Secrets - SuperLink: mount server cert, switch from --insecure to --ssl-* flags - SuperNode: distribute CA cert via AddOnTemplate Secret, use --root-certificates - Add tls.enabled toggle (default: false) for backward compatibility - Helm lookup with fail guard ensures clear error if certs not generated Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Meng Yan <myan@redhat.com> * 📖 Add flwr run TLS configuration docs Add instructions for configuring TLS in ~/.flwr/config.toml when submitting FL jobs via flwr run to the TLS-secured SuperLink control API. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Meng Yan <myan@redhat.com> * 📖 Move TLS docs to docs/enable-tls.md Move the Enable TLS section from README into a dedicated doc file under docs/ for consistency with other guides, and reference it from the Roadmap section. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Meng Yan <myan@redhat.com> * 🐛 Add server-cert Secret validation and clarify namespace in TLS docs - Add lookup/fail guard for server-cert Secret in superlink.yaml to fail fast when the Secret is missing (mirrors existing CA Secret validation) - Update TLS docs to clarify namespace assumptions and guide users to pass matching --namespace when overriding superlink.namespace Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Meng Yan <myan@redhat.com> --------- Signed-off-by: Meng Yan <myan@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c6e505e commit dce6471

6 files changed

Lines changed: 267 additions & 1 deletion

File tree

flower-addon/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ Flower Addon leverages OCM's multi-cluster management to address these challenge
5858
- [x] [Auto-Install with Placement](docs/auto-install-by-placement.md) - Schedule SuperNodes across clusters via OCM Placement ([demo](https://asciinema.org/a/776746))
5959
- [x] [Run Federated Learning Applications](docs/run-federated-app.md) - Run federated learning applications on the Flower Addon environment ([demo](https://asciinema.org/a/776749))
6060
- [x] Automatic ClientApp distribution via ManifestWorkReplicaSet
61-
- [ ] TLS-secured SuperNode-SuperLink connections via Addon auto-registration
61+
- [x] [Enable TLS](docs/enable-tls.md) - TLS-secured SuperNode-SuperLink connections
6262

6363
## Related Projects
6464

flower-addon/charts/flower-addon/templates/addon-template.yaml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,12 +46,22 @@ spec:
4646
image: "{{`{{IMAGE}}`}}"
4747
imagePullPolicy: {{ .Values.supernode.image.pullPolicy }}
4848
args:
49+
{{- if .Values.tls.enabled }}
50+
- "--root-certificates=/etc/flower/tls/ca.crt"
51+
{{- else }}
4952
- "--insecure"
53+
{{- end }}
5054
- "--superlink={{`{{SUPERLINK_ADDRESS}}`}}:{{`{{SUPERLINK_PORT}}`}}"
5155
- "--node-config"
5256
- 'cluster-name="{{`{{CLUSTER_NAME}}`}}" num-partitions={{`{{NUM_PARTITIONS}}`}}'
5357
- "--isolation=process"
5458
- "--clientappio-api-address=0.0.0.0:9094"
59+
{{- if .Values.tls.enabled }}
60+
volumeMounts:
61+
- name: tls-ca
62+
mountPath: /etc/flower/tls
63+
readOnly: true
64+
{{- end }}
5565
ports:
5666
- name: clientappio
5767
containerPort: 9094
@@ -68,6 +78,28 @@ spec:
6878
port: 9094
6979
initialDelaySeconds: 5
7080
periodSeconds: 5
81+
{{- if .Values.tls.enabled }}
82+
volumes:
83+
- name: tls-ca
84+
secret:
85+
secretName: flower-tls-ca
86+
{{- end }}
87+
88+
{{- if .Values.tls.enabled }}
89+
# CA certificate Secret (distributed to managed clusters)
90+
{{- $caSecret := (lookup "v1" "Secret" .Values.superlink.namespace .Values.tls.ca.secretName) }}
91+
{{- if not $caSecret }}
92+
{{- fail (printf "TLS enabled but Secret '%s/%s' not found. Run hack/generate-certs.sh first." .Values.superlink.namespace .Values.tls.ca.secretName) }}
93+
{{- end }}
94+
- apiVersion: v1
95+
kind: Secret
96+
metadata:
97+
name: flower-tls-ca
98+
namespace: flower-addon
99+
type: Opaque
100+
data:
101+
ca.crt: {{ index $caSecret.data "ca.crt" }}
102+
{{- end }}
71103

72104
# SuperNode Service
73105
- apiVersion: v1

flower-addon/charts/flower-addon/templates/superlink.yaml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,10 @@
11
{{- if .Values.superlink.enabled }}
2+
{{- if .Values.tls.enabled }}
3+
{{- $serverSecret := (lookup "v1" "Secret" .Values.superlink.namespace .Values.tls.serverCert.secretName) }}
4+
{{- if not $serverSecret }}
5+
{{- fail (printf "TLS enabled but Secret '%s/%s' not found. Run hack/generate-certs.sh first." .Values.superlink.namespace .Values.tls.serverCert.secretName) }}
6+
{{- end }}
7+
{{- end }}
28
apiVersion: apps/v1
39
kind: Deployment
410
metadata:
@@ -21,9 +27,16 @@ spec:
2127
image: {{ .Values.superlink.image.repository }}:{{ .Values.superlink.image.tag }}
2228
imagePullPolicy: {{ .Values.superlink.image.pullPolicy }}
2329
args:
30+
{{- if .Values.tls.enabled }}
31+
- "--ssl-ca-certfile=/etc/flower/tls/ca.crt"
32+
- "--ssl-certfile=/etc/flower/tls/server.pem"
33+
- "--ssl-keyfile=/etc/flower/tls/server.key"
34+
{{- end }}
2435
{{- range .Values.superlink.args }}
36+
{{- if not (and $.Values.tls.enabled (eq . "--insecure")) }}
2537
- {{ . | quote }}
2638
{{- end }}
39+
{{- end }}
2740
ports:
2841
- name: exec
2942
containerPort: 9091
@@ -34,6 +47,12 @@ spec:
3447
- name: control
3548
containerPort: 9093
3649
protocol: TCP
50+
{{- if .Values.tls.enabled }}
51+
volumeMounts:
52+
- name: tls-certs
53+
mountPath: /etc/flower/tls
54+
readOnly: true
55+
{{- end }}
3756
resources:
3857
{{- toYaml .Values.superlink.resources | nindent 10 }}
3958
livenessProbe:
@@ -46,6 +65,12 @@ spec:
4665
port: 9092
4766
initialDelaySeconds: 5
4867
periodSeconds: 5
68+
{{- if .Values.tls.enabled }}
69+
volumes:
70+
- name: tls-certs
71+
secret:
72+
secretName: {{ .Values.tls.serverCert.secretName }}
73+
{{- end }}
4974
---
5075
apiVersion: v1
5176
kind: Service

flower-addon/charts/flower-addon/values.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,14 @@ superlink:
2323
- "--insecure"
2424
- "--isolation=process"
2525

26+
# TLS configuration
27+
tls:
28+
enabled: false
29+
ca:
30+
secretName: flower-tls-ca # CA secret (created by hack/generate-certs.sh)
31+
serverCert:
32+
secretName: flower-superlink-tls # Server cert secret
33+
2634
# SuperNode configuration (deployed on managed clusters via OCM addon)
2735
supernode:
2836
image:

flower-addon/docs/enable-tls.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Enable TLS
2+
3+
By default, SuperLink and SuperNode communicate over insecure gRPC. This guide enables TLS-secured connections between them.
4+
5+
**What gets configured:**
6+
- **SuperLink** on hub cluster starts with `--ssl-certfile`/`--ssl-keyfile` for server-side TLS
7+
- **SuperNode** on managed clusters connects with `--root-certificates` to verify the SuperLink identity
8+
- **CA certificate** is automatically distributed to managed clusters via the OCM AddOnTemplate
9+
10+
## 1. Generate Certificates
11+
12+
```bash
13+
cd flower-addon
14+
15+
# Generate CA + server cert (include hub node IP in SANs for NodePort access)
16+
./hack/generate-certs.sh --hub-ip <HUB_NODE_IP>
17+
```
18+
19+
This creates two Kubernetes Secrets in `flower-system` (default namespace). If you override `superlink.namespace` in Helm values, pass the same namespace here:
20+
21+
```bash
22+
./hack/generate-certs.sh --hub-ip <HUB_NODE_IP> --namespace <YOUR_NAMESPACE>
23+
```
24+
25+
- `flower-tls-ca` — CA certificate and key (used to verify server identity)
26+
- `flower-superlink-tls` — SuperLink server certificate, key, and CA cert
27+
28+
The server certificate SANs include:
29+
- `superlink`, `superlink.flower-system`, `superlink.flower-system.svc`, `superlink.flower-system.svc.cluster.local`
30+
- Any hub node IPs passed via `--hub-ip` (repeatable for multiple IPs)
31+
32+
## 2. Deploy with TLS Enabled
33+
34+
```bash
35+
helm install flower-addon ./charts/flower-addon \
36+
--set tls.enabled=true \
37+
--set deploymentConfig.superlinkAddress=<HUB_NODE_IP>
38+
```
39+
40+
When `tls.enabled=true`:
41+
- SuperLink mounts the server cert secret and starts with `--ssl-ca-certfile`, `--ssl-certfile`, `--ssl-keyfile`
42+
- The CA certificate is read from the `flower-tls-ca` Secret (via Helm `lookup`) and embedded in the AddOnTemplate
43+
- SuperNodes on managed clusters receive the CA cert as a Secret and start with `--root-certificates`
44+
45+
## 3. Run FL Jobs with TLS
46+
47+
When submitting FL jobs via `flwr run`, configure TLS in `~/.flwr/config.toml`:
48+
49+
```toml
50+
[superlink.ocm-deployment]
51+
address = "<HUB_NODE_IP>:30093"
52+
insecure = false
53+
root-certificates = "/path/to/ca.crt"
54+
```
55+
56+
Extract the CA cert from the cluster:
57+
58+
```bash
59+
# Use the same namespace as your SuperLink deployment (default: flower-system)
60+
kubectl get secret flower-tls-ca -n flower-system -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
61+
```
62+
63+
Then run:
64+
65+
```bash
66+
flwr run . ocm-deployment --stream
67+
```
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
#!/usr/bin/env bash
2+
# Generate TLS certificates for Flower SuperLink and create Kubernetes Secrets.
3+
#
4+
# Usage:
5+
# ./hack/generate-certs.sh [--hub-ip <IP>] [--namespace <ns>] [--days <validity>]
6+
#
7+
# This creates two Secrets:
8+
# flower-tls-ca - CA cert + key (used to verify server identity)
9+
# flower-superlink-tls - SuperLink server cert + key + CA cert
10+
11+
set -euo pipefail
12+
13+
NAMESPACE="${NAMESPACE:-flower-system}"
14+
DAYS=365
15+
HUB_IPS=()
16+
CERT_DIR=""
17+
18+
usage() {
19+
echo "Usage: $0 [--hub-ip <IP>]... [--namespace <ns>] [--days <n>]"
20+
echo ""
21+
echo "Options:"
22+
echo " --hub-ip <IP> Hub node IP to include in server cert SANs (repeatable)"
23+
echo " --namespace <ns> Kubernetes namespace (default: flower-system)"
24+
echo " --days <n> Certificate validity in days (default: 365)"
25+
exit 1
26+
}
27+
28+
while [[ $# -gt 0 ]]; do
29+
case "$1" in
30+
--hub-ip)
31+
[[ -z "${2:-}" ]] && { echo "Error: --hub-ip requires a value"; exit 1; }
32+
HUB_IPS+=("$2")
33+
shift 2
34+
;;
35+
--namespace)
36+
[[ -z "${2:-}" ]] && { echo "Error: --namespace requires a value"; exit 1; }
37+
NAMESPACE="$2"
38+
shift 2
39+
;;
40+
--days)
41+
[[ -z "${2:-}" ]] && { echo "Error: --days requires a value"; exit 1; }
42+
DAYS="$2"
43+
shift 2
44+
;;
45+
-h|--help)
46+
usage
47+
;;
48+
*)
49+
echo "Unknown option: $1"
50+
usage
51+
;;
52+
esac
53+
done
54+
55+
cleanup() {
56+
if [[ -n "${CERT_DIR}" && -d "${CERT_DIR}" ]]; then
57+
rm -rf "${CERT_DIR}"
58+
fi
59+
}
60+
trap cleanup EXIT
61+
62+
CERT_DIR=$(mktemp -d)
63+
64+
echo "==> Generating CA certificate..."
65+
openssl req -x509 -newkey rsa:2048 -nodes \
66+
-keyout "${CERT_DIR}/ca.key" \
67+
-out "${CERT_DIR}/ca.crt" \
68+
-days "${DAYS}" \
69+
-subj "/CN=Flower CA/O=flower-addon"
70+
71+
echo "==> Generating SuperLink server certificate..."
72+
73+
# Build SAN extension
74+
SAN="DNS:superlink,DNS:superlink.${NAMESPACE},DNS:superlink.${NAMESPACE}.svc,DNS:superlink.${NAMESPACE}.svc.cluster.local"
75+
for ip in "${HUB_IPS[@]}"; do
76+
SAN="${SAN},IP:${ip}"
77+
done
78+
79+
cat > "${CERT_DIR}/server-ext.cnf" <<EOF
80+
[req]
81+
req_extensions = v3_req
82+
distinguished_name = req_distinguished_name
83+
84+
[req_distinguished_name]
85+
86+
[v3_req]
87+
basicConstraints = CA:FALSE
88+
keyUsage = digitalSignature, keyEncipherment
89+
extendedKeyUsage = serverAuth
90+
subjectAltName = ${SAN}
91+
EOF
92+
93+
openssl req -newkey rsa:2048 -nodes \
94+
-keyout "${CERT_DIR}/server.key" \
95+
-out "${CERT_DIR}/server.csr" \
96+
-subj "/CN=superlink/O=flower-addon"
97+
98+
openssl x509 -req \
99+
-in "${CERT_DIR}/server.csr" \
100+
-CA "${CERT_DIR}/ca.crt" \
101+
-CAkey "${CERT_DIR}/ca.key" \
102+
-CAcreateserial \
103+
-out "${CERT_DIR}/server.pem" \
104+
-days "${DAYS}" \
105+
-extensions v3_req \
106+
-extfile "${CERT_DIR}/server-ext.cnf"
107+
108+
echo "==> Creating namespace ${NAMESPACE} (if not exists)..."
109+
kubectl create namespace "${NAMESPACE}" --dry-run=client -o yaml | kubectl apply -f -
110+
111+
echo "==> Creating Secret flower-tls-ca..."
112+
kubectl create secret generic flower-tls-ca \
113+
--namespace="${NAMESPACE}" \
114+
--from-file=ca.crt="${CERT_DIR}/ca.crt" \
115+
--from-file=ca.key="${CERT_DIR}/ca.key" \
116+
--dry-run=client -o yaml | kubectl apply -f -
117+
118+
echo "==> Creating Secret flower-superlink-tls..."
119+
kubectl create secret generic flower-superlink-tls \
120+
--namespace="${NAMESPACE}" \
121+
--from-file=server.pem="${CERT_DIR}/server.pem" \
122+
--from-file=server.key="${CERT_DIR}/server.key" \
123+
--from-file=ca.crt="${CERT_DIR}/ca.crt" \
124+
--dry-run=client -o yaml | kubectl apply -f -
125+
126+
echo ""
127+
echo "TLS certificates created successfully!"
128+
echo " Namespace: ${NAMESPACE}"
129+
echo " CA Secret: flower-tls-ca"
130+
echo " TLS Secret: flower-superlink-tls"
131+
echo " SANs: ${SAN}"
132+
echo " Validity: ${DAYS} days"
133+
echo ""
134+
echo "Next: helm install flower-addon ./charts/flower-addon --set tls.enabled=true ..."

0 commit comments

Comments
 (0)