Skip to content

Commit 22651e2

Browse files
authored
fix(recipes): raise ebs-csi node sidecar memory limits for p5 nodes (NVIDIA#1693)
Signed-off-by: Nathan Hensley <nhensley@nvidia.com>
1 parent e8450d5 commit 22651e2

1 file changed

Lines changed: 28 additions & 0 deletions

File tree

recipes/components/aws-ebs-csi-driver/values.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,34 @@ node:
5454
rollingUpdate:
5555
maxUnavailable: "10%"
5656

57+
# Sidecar memory limits. The chart defaults (32Mi) are too tight for the node
58+
# DaemonSet's livenessProbe / node-driver-registrar containers on high-core
59+
# instances (e.g. p5.48xlarge, 192 vCPU). The Go binary's resident set alone is
60+
# ~24Mi (file-rss); adding per-P runtime overhead (Go sizes runtime structures
61+
# by GOMAXPROCS, which defaults to the host core count) pushes the working set
62+
# to ~31.7Mi during cold start under bringup load, cgroup-OOM-killing
63+
# livenessProbe (observed: CONSTRAINT_MEMCG, RSS 31.7Mi vs 32Mi). Killing the
64+
# livenessProbe removes the /healthz endpoint the ebs-plugin livenessProbe
65+
# depends on, so kubelet restarts ebs-plugin -> CrashLoopBackOff -> the
66+
# aws-ebs-csi-driver health check trips -> expected-resources (deployment
67+
# validation) fails. 128Mi gives headroom above the observed peak and is
68+
# independent of instance core count.
69+
sidecars:
70+
nodeDriverRegistrar:
71+
resources:
72+
requests:
73+
cpu: 10m
74+
memory: 40Mi
75+
limits:
76+
memory: 128Mi
77+
livenessProbe:
78+
resources:
79+
requests:
80+
cpu: 10m
81+
memory: 40Mi
82+
limits:
83+
memory: 128Mi
84+
5785
storageClasses: []
5886

5987
# Provision a cluster-default StorageClass (ebs-csi-default-sc: gp3,

0 commit comments

Comments
 (0)