Skip to content

--enable-csi-node-aware-scheduling blocks scale-up from zero #10131

Description

@melissa-defore

Which component are you using?: /area cluster-autoscaler

What version of the component are you using?: 1.35.0

Component version:

What k8s version are you using (kubectl version)?: 1.35.6

kubectl version Output
$ kubectl version
Client Version: v1.36.1
Kustomize Version: v5.8.1
Server Version: v1.35.6

What environment is this in?: EKS

What did you expect to happen?:
With --enable-csi-node-aware-scheduling enabled, a pending pod that needs a CSI volume should trigger scale-up of an empty AWS node group (managed node group at size 0). Cluster autoscaler should build a usable template NodeInfo for that group and increase the group size.

What happened instead?:
Scale-up does not trigger. The pod stays pending with NotTriggerScaleUp, and cluster autoscaler reports that it cannot obtain NodeInfo for the template node because no CSINode exists for it.

Normal NotTriggerScaleUp ... cluster-autoscaler pod didn't trigger scale-up:
1 can't schedule pod <ns>/<pod>: unexpected error: error obtaining NodeInfo for name
"template-node-for-<asg>-...": csi nodes template-node-for-<asg>-... not found

The same node group scales correctly for that pod when it already has at least one live node.

How to reproduce it (as minimally and precisely as possible):

  1. EKS cluster with: EBS CSI driver installed and cluster autoscaler on 1.35.0 with --enable-csi-node-aware-scheduling=true
  2. Create (or use) a managed node group / ASG that: can schedule the workload (labels/taints match) and has min size=0 (fully scaled to zero)
  3. Create a PVC using an EBS storage class (e.g. gp3 / in-tree EBS CSI, whatever) and a pod that mounts it (optionally with a node selector for that empty group).
  4. Observe the Pod remain Pending and emit NotTriggerScaleUp with 'csi nodes template-node-for-... not found' error noted earlier.
  5. Set the same node group min size ≥ 1, wait for a node to become Ready, then recreate the pending Pod (or leave it pending): scale-up / scheduling for CSI-volume pods succeeds

Anything else we need to know?:
Nope

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cluster-autoscalerIssues or PRs related to the Cluster Autoscaler componentkind/bugCategorizes issue or PR as related to a bug.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions