The ValkeyReconciler creates a StatefulSet with controllerutil.SetControllerReference (setting the Valkey CR as the owner), but SetupWithManager only registers a watch on the Valkey custom resource itself via For(&hyperv1.Valkey{}). It does not call Owns(&appsv1.StatefulSet{}), so changes to the owned StatefulSet — including pod deletions, PVC losses, or replica count drift — do not trigger a reconciliation.
Root Cause
The upstream SetupWithManager at internal/controller/valkey_controller.go line 2768:
func (r *ValkeyReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&hyperv1.Valkey{}).
Complete(r)
}
The controller only watches the Valkey CR and the changes on the stateful set are not reconciled.
Impact
When a pod or PVC belonging to the Valkey StatefulSet is deleted (e.g., due to node failure, manual intervention, or storage issues):
- No reconciliation is triggered. The operator does not notice that the StatefulSet's actual state has diverged from the desired state.
- The StatefulSet controller (kube-controller-manager) will recreate the pod, but the new pod comes up with a fresh Valkey instance — empty data, no cluster membership, and a new node ID.
- The Valkey cluster remains degraded because the operator doesn't run
initCluster to re-meet the new node into the cluster, reassign slots, or reconfigure replication.
- Manual intervention is required — an operator restart or a no-op edit to the
Valkey CR is needed to force a reconcile and heal the cluster.
This causes issues in scenarios like:
- Node eviction / drain: Pods are rescheduled, PVCs may be lost if using local storage, and the cluster needs re-initialization.
- PVC deletion: The StatefulSet recreates the pod with a new PVC, but without reconciliation the operator never detects the data loss or reconfigures the cluster topology.
Fix
Add Owns(&appsv1.StatefulSet{}) to SetupWithManager:
func (r *ValkeyReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&hyperv1.Valkey{}).
Owns(&appsv1.StatefulSet{}).
Complete(r)
}
The
ValkeyReconcilercreates aStatefulSetwithcontrollerutil.SetControllerReference(setting theValkeyCR as the owner), butSetupWithManageronly registers a watch on theValkeycustom resource itself viaFor(&hyperv1.Valkey{}). It does not callOwns(&appsv1.StatefulSet{}), so changes to the owned StatefulSet — including pod deletions, PVC losses, or replica count drift — do not trigger a reconciliation.Root Cause
The upstream
SetupWithManageratinternal/controller/valkey_controller.goline 2768:The controller only watches the
ValkeyCR and the changes on the stateful set are not reconciled.Impact
When a pod or PVC belonging to the Valkey StatefulSet is deleted (e.g., due to node failure, manual intervention, or storage issues):
initClusterto re-meet the new node into the cluster, reassign slots, or reconfigure replication.ValkeyCR is needed to force a reconcile and heal the cluster.This causes issues in scenarios like:
Fix
Add
Owns(&appsv1.StatefulSet{})toSetupWithManager: