Skip to content

refactor: use deferred status patch in ValkeyNode controller - #274

Closed
daanvinken wants to merge 1 commit into
valkey-io:mainfrom
daanvinken:refactor/valkeynode-deferred-status
Closed

refactor: use deferred status patch in ValkeyNode controller#274
daanvinken wants to merge 1 commit into
valkey-io:mainfrom
daanvinken:refactor/valkeynode-deferred-status

Conversation

@daanvinken

@daanvinken daanvinken commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Replace scattered Status().Patch calls in the ValkeyNode controller with a single deferred patch at the end of Reconcile, following the common pattern in operators like CNPG and Strimzi. Idea was discussed at #257 (comment).

Implementation

  • Snapshot node at the top of Reconcile with client.MergeFrom(node.DeepCopy())
  • Local closures mutate node.Status in memory
  • Single defer flushes the patch, with named return values to propagate errors
  • Eliminates setReadyCondition, setLiveConfigCondition, clearLiveConfigCondition, updateStatus

Testing

Unit tests pass. Added Reconcile-level tests for Ready=false when no pod exists and idempotency.

Replace scattered Status().Patch calls with a single deferred patch at
the end of Reconcile. All condition mutations happen in memory via
local closures (setCondition, removeCondition, setReady) and are
flushed by the defer.

This eliminates:
- setReadyCondition (re-fetched + patched on every error)
- setLiveConfigCondition (re-fetched + patched per call)
- clearLiveConfigCondition (re-fetched + patched per call)
- updateStatus (re-fetched + patched per call)

The deferred pattern is the common approach in Kubernetes operators
(e.g. CNPG, Strimzi) and reduces API calls from up to 4 per reconcile
to exactly 1. A follow-up PR will apply the same pattern to the
ValkeyCluster controller.

Signed-off-by: Daan Vinken <daanvinken@tythus.com>
@daanvinken

Copy link
Copy Markdown
Contributor Author

Closing this for now. After looking at the diff more carefully, I don't think the refactor justifies itself at the current codebase size. With only 4 status update paths, the old code with #257's setReadyCondition fix is already clear and each helper method is independently testable.

The deferred pattern adds closures, named return values, and error merging in the defer, which makes the Reconcile function harder to follow without solving a real problem at this scale. The extra API calls (4 vs 1 per reconcile) are negligible.

If the controller grows significantly more status update paths in the future, this pattern would make more sense. For now the current code is fine.

@daanvinken daanvinken closed this Jun 23, 2026
@greptile-apps

greptile-apps Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR replaces scattered Status().Patch() calls in the ValkeyNode controller with a single deferred patch at the end of Reconcile, following the common pattern seen in CNPG and Strimzi. A patchBase snapshot is taken via client.MergeFrom(node.DeepCopy()) immediately after fetching the node, local closures mutate node.Status in memory, and a single defer flushes the patch on any exit path—eliminating the old setReadyCondition, setLiveConfigCondition, clearLiveConfigCondition, and updateStatus helpers.

  • The deferred patch is set up before reconcilePersistenceFinalizer, so it fires on every finalizer-requeue cycle with an empty diff, adding an extra sub-resource API call that was absent before.
  • node.Status.ObservedGeneration is stamped before getPersistentVolumeClaim and getPod are called; if either of those returns an error, the deferred patch persists ObservedGeneration = node.Generation without a corresponding Ready-condition update, signalling false completion to downstream consumers.
  • The explicit reflect.DeepEqual no-op guard from updateStatus is gone; idempotency now relies on the API server treating empty merge-patch bodies as no-ops.

Confidence Score: 3/5

The core reconciliation logic is sound and the deferred-patch pattern is well-established, but the premature ObservedGeneration write on partial failures could cause downstream controllers to treat an incompletely reconciled node as fully processed.

The ordering of node.Status.ObservedGeneration = node.Generation before the getPersistentVolumeClaim and getPod calls means that API errors from either function will cause the deferred patch to commit ObservedGeneration to the server without setting a Ready condition—a state the old updateStatus never produced.

The ordering change in valkeynode_controller.go around ObservedGeneration and the early-error returns after line 177 deserves close attention before merging.

Important Files Changed

Filename Overview
internal/controller/valkeynode_controller.go Refactors scattered Status().Patch() calls into a single deferred patch; introduces a subtle ordering issue where ObservedGeneration is set before PVC/pod fetch errors can be caught, causing it to be persisted even on mid-reconcile failures.
internal/controller/valkeynode_controller_test.go Tests updated from unit-level helper tests to full Reconcile-level integration tests; idempotency assertion weakened (failure message removed, explicit skip guarantee replaced with API-server no-op reliance).

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CR as Controller Runtime
    participant R as Reconcile()
    participant API as Kubernetes API
    participant Defer as deferred patch

    CR->>R: Reconcile(ctx, req)
    R->>API: Get ValkeyNode (node)
    Note over R: patchBase = MergeFrom(node.DeepCopy())
    Note over R: defer Status().Patch(node, patchBase)

    R->>API: reconcilePersistenceFinalizer (may Update node)
    alt finalizer added/removed
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (empty diff)
    end

    R->>API: ensureConfigMap / ensurePVC / ensureWorkload
    alt error
        Note over R: setCondition(Ready=False)
        R-->>Defer: early return triggers defer
        Defer->>API: "Status().Patch (Ready=False)"
    end

    Note over R: ObservedGeneration = node.Generation
    R->>API: getPersistentVolumeClaim
    alt error
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (ObservedGeneration only)
    end
    R->>API: getPod
    alt error
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (ObservedGeneration + PVC conds)
    end

    Note over R: Update node.Status (Ready, Role, conditions)
    R->>API: applyLiveConfig
    Note over R: setCondition(LiveConfigApplied)
    R-->>Defer: normal return triggers defer
    Defer->>API: Status().Patch (full status)
    R-->>CR: result, retErr
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CR as Controller Runtime
    participant R as Reconcile()
    participant API as Kubernetes API
    participant Defer as deferred patch

    CR->>R: Reconcile(ctx, req)
    R->>API: Get ValkeyNode (node)
    Note over R: patchBase = MergeFrom(node.DeepCopy())
    Note over R: defer Status().Patch(node, patchBase)

    R->>API: reconcilePersistenceFinalizer (may Update node)
    alt finalizer added/removed
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (empty diff)
    end

    R->>API: ensureConfigMap / ensurePVC / ensureWorkload
    alt error
        Note over R: setCondition(Ready=False)
        R-->>Defer: early return triggers defer
        Defer->>API: "Status().Patch (Ready=False)"
    end

    Note over R: ObservedGeneration = node.Generation
    R->>API: getPersistentVolumeClaim
    alt error
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (ObservedGeneration only)
    end
    R->>API: getPod
    alt error
        R-->>Defer: early return triggers defer
        Defer->>API: Status().Patch (ObservedGeneration + PVC conds)
    end

    Note over R: Update node.Status (Ready, Role, conditions)
    R->>API: applyLiveConfig
    Note over R: setCondition(LiveConfigApplied)
    R-->>Defer: normal return triggers defer
    Defer->>API: Status().Patch (full status)
    R-->>CR: result, retErr
Loading

Reviews (1): Last reviewed commit: "refactor: use deferred status patch in V..." | Re-trigger Greptile

Comment on lines +177 to 195
node.Status.ObservedGeneration = node.Generation

pvc, err := r.getPersistentVolumeClaim(ctx, node)
if err != nil {
return ctrl.Result{}, err
}
if node.Spec.Persistence != nil {
pvcStatus, pvcReason, pvcMessage := pvcStatusCondition(pvc)
setCondition(valkeyiov1alpha1.ValkeyNodeConditionPersistentVolumeClaimReady, pvcStatus, pvcReason, pvcMessage)
pvcSizeStatus, pvcSizeReason, pvcSizeMessage := pvcSizeStatusCondition(node, pvc)
setCondition(valkeyiov1alpha1.ValkeyNodeConditionPersistentVolumeClaimSizeReady, pvcSizeStatus, pvcSizeReason, pvcSizeMessage)
} else {
removeCondition(valkeyiov1alpha1.ValkeyNodeConditionPersistentVolumeClaimReady)
removeCondition(valkeyiov1alpha1.ValkeyNodeConditionPersistentVolumeClaimSizeReady)
}

if err := r.updateStatus(ctx, node); err != nil {
pod, err := r.getPod(ctx, node)
if err != nil {
return ctrl.Result{}, err

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 ObservedGeneration committed before error-path returns

node.Status.ObservedGeneration is stamped at line 177, before the getPersistentVolumeClaim (line 179) and getPod (line 193) calls. When either of those returns an error, the reconcile exits early but the deferred patch still fires—writing ObservedGeneration = node.Generation to the API server without a corresponding Ready condition update.

In the old updateStatus implementation, an error from getPersistentVolumeClaim caused the function to return before any patch was issued, so ObservedGeneration was never advanced on partial failures. Any downstream controller (e.g., ValkeyCluster) that reads node.Status.ObservedGeneration == node.Generation as "this generation has been fully reconciled" can draw an incorrect conclusion during the window between the failed reconcile and the next retry.

Move the ObservedGeneration assignment to after the PVC and Pod are successfully fetched, or gate it behind the success path only.

Comment on lines +123 to +130
defer func() {
if err := r.Status().Patch(ctx, node, patchBase); err != nil {
log.Error(err, "failed to patch ValkeyNode status")
if retErr == nil {
retErr = err
}
}
}()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Deferred patch fires on reconcilePersistenceFinalizer requeue

When reconcilePersistenceFinalizer adds or removes the cleanup finalizer and returns requeue=true (lines 146–150), the function exits before any status mutation has occurred. The deferred Status().Patch() still runs at that point, sending a no-op diff to the API server on every finalizer-transition cycle.

This is an extra sub-resource API call per requeue that was absent in the old code. Consider moving the patchBase snapshot and defer registration to after the reconcilePersistenceFinalizer early returns.

Comment on lines +120 to +130
// Snapshot for deferred status patch.
patchBase := client.MergeFrom(node.DeepCopy())

defer func() {
if err := r.Status().Patch(ctx, node, patchBase); err != nil {
log.Error(err, "failed to patch ValkeyNode status")
if retErr == nil {
retErr = err
}
}
}()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Explicit no-op guard removed; idempotency now relies on API server behaviour

The old updateStatus used reflect.DeepEqual(patchBase.Status, current.Status) to skip the Status().Patch() call when nothing changed. That explicit guard is gone. Every reconcile now unconditionally issues a status patch, relying on the Kubernetes API server to not bump ResourceVersion when the JSON merge-patch body resolves to an empty diff.

The corresponding test was also weakened—it dropped the "status write should be skipped when nothing changed" failure message and no longer asserts the skip, only that ResourceVersion is equal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant