Skip to content

Operator fails to reconcile on non-GCP clusters when gcpConfigConnector is not set (v2.3.0) #103

Description

@twellck

In v2.3.0, the operator unconditionally attempts to ensure IAMPolicyMember resources do not exist during RestateCluster reconciliation, even when gcpConfigConnector is not set in the Helm values. However, the RBAC rule for iam.cnrm.cloud.google.com/iampolicymembers is conditionally included only when gcpConfigConnector is truthy.

This causes the operator to fail with a 403 Forbidden error and enter a reconcile loop on non-GCP clusters (e.g., AWS EKS):

reconcile failed: ApplyFailed(KubeError(Api(ErrorResponse { 
  status: "Failure", 
  message: "iampolicymembers.iam.cnrm.cloud.google.com \"restate-workload-identity\" is forbidden: 
    User \"system:serviceaccount:restate-operator:restate-operator\" cannot delete resource 
    \"iampolicymembers\" in API group \"iam.cnrm.cloud.google.com\" in the namespace \"restate\"", 
  reason: "Forbidden", 
  code: 403 
})))

Expected behavior:

The operator should either:

  1. Skip the IAMPolicyMember cleanup when gcpConfigConnector is not configured (match the RBAC condition), or
  2. Always include the RBAC rule regardless of the gcpConfigConnector value (so the cleanup gets a 404 instead of 403)

Workaround:

Set gcpConfigConnector: true in Helm values to include the RBAC rule. The operator will then get a 404 (resource not found) instead of 403, and reconciliation succeeds.

Environment:

  • Operator version: 2.3.0
  • Chart version: 2.3.0
  • Cluster: AWS EKS (non-GCP)
  • gcpConfigConnector: not set (null)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions