-
Notifications
You must be signed in to change notification settings - Fork 4.7k
[hetzner] cluster-autoscaler addon missing HCLOUD_CLUSTER_CONFIG #18136
Description
Summary
The kops-managed cluster-autoscaler addon for Hetzner does not set the HCLOUD_CLUSTER_CONFIG (or legacy HCLOUD_CLOUD_INIT) environment variable. The Hetzner autoscaler cloud provider requires at least one of these to be present — without them it refuses to start, logging:
neither `HCLOUD_CLUSTER_CONFIG`, `HCLOUD_CLOUD_INIT` nor `HCLOUD_CLUSTER_CONFIG_FILE` is specified
Background
The Hetzner cluster-autoscaler cloud provider uses HCLOUD_CLUSTER_CONFIG (a base64-encoded JSON blob) to configure per-node-group behaviour: the image to use when creating new servers, per-group cloud-init scripts, and per-group Hetzner server labels.
kops already stamps a rich set of labels on servers it creates directly (cluster name, instance group, role, node labels). Without providing this config to the autoscaler, servers it creates will:
- Not bootstrap correctly (no cloud-init)
- Not carry the kops labels required for cloud instance group reconciliation
Item 2 is addressed on the autoscaler side by kubernetes/autoscaler#9430, which propagates the nodeConfig.labels from HCLOUD_CLUSTER_CONFIG onto the Hetzner server at creation time.
What needs to be done
kops must generate a HCLOUD_CLUSTER_CONFIG value and pass it to the autoscaler pod. The config must contain:
Labels (straightforward)
The same labels applied by CloudTagsForInstanceGroup() — cluster name, instance group, instance role, and all node labels. These are already computed during kops update cluster.
Per-group image (straightforward)
The Hetzner server image name, from ig.Spec.Image.
Cloud-init / nodeup bootstrap script (hard)
The nodeup shell script that bootstraps a Kubernetes node. Generating this requires:
- CA keypairs (available in the keystore, but not currently threaded through
TemplateFunctions) - URLs of the nodeup binary (available as
NodeUpAssetsinapply_cluster.go, but not passed to the template layer) - The
NodeupConfigHash(SHA256 of the nodeupconfig.yaml, computed during bootstrap-script task execution)
This is the blocking piece. Two implementation paths are possible:
Option A — thread through TemplateFunctions: Pass the keystore, NodeUpAssets, and NodeUpConfigBuilder to TemplateFunctions and compute the script at addon-template render time. The NodeupConfigHash would need to be computed separately at render time before the tasks run.
Option B — dedicated post-build task: Create a new HetznerClusterAutoscalerConfigSecret task that depends on the BootstrapScript tasks, reads their rendered cloud-init content, assembles the JSON, and writes it to a Kubernetes Secret. This is more architecturally correct but requires more scaffolding.
Relationship to other issues
- Completes the work started in Cluster Autoscaling for Hetzner #17543 (missing
HCLOUD_TOKEN) and the companion PRs fixing the addon env vars and--nodesformat. - Requires feat(hetzner): add serverLabels field to nodeConfig for Hetzner server labels autoscaler#9430 (or equivalent) to be merged so that the labels in
HCLOUD_CLUSTER_CONFIGare actually applied to new Hetzner servers.