Skip to content

Commit 6705e42

Browse files
committed
fix(entrypoint): add /usr/local/nvidia to PATH and ldconfig on GKE
GKE container runtime injects /usr/local/nvidia from the host but does not add it to PATH or run ldconfig, unlike the nvidia-container-runtime. This caused nvidia-smi not found errors on GCP since v1.9.0 introduced the explicit nvidia-smi check in the entrypoints.
1 parent 5bc4d88 commit 6705e42

2 files changed

Lines changed: 20 additions & 0 deletions

File tree

cuda-all-entrypoint.sh

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
#!/bin/bash
22

3+
# GKE injects /usr/local/nvidia from the host but does not add it to PATH or
4+
# run ldconfig for it, unlike the nvidia-container-runtime on AWS which does both.
5+
if [ -d /usr/local/nvidia/bin ]; then
6+
export PATH="${PATH}:/usr/local/nvidia/bin"
7+
fi
8+
if [ -d /usr/local/nvidia/lib64 ]; then
9+
echo /usr/local/nvidia/lib64 > /etc/ld.so.conf.d/nvidia-host.conf
10+
ldconfig
11+
fi
12+
313
if ! command -v nvidia-smi &>/dev/null; then
414
echo "Error: 'nvidia-smi' command not found."
515
exit 1

cuda-entrypoint.sh

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
#!/bin/bash
22

3+
# GKE injects /usr/local/nvidia from the host but does not add it to PATH or
4+
# run ldconfig for it, unlike the nvidia-container-runtime on AWS which does both.
5+
if [ -d /usr/local/nvidia/bin ]; then
6+
export PATH="${PATH}:/usr/local/nvidia/bin"
7+
fi
8+
if [ -d /usr/local/nvidia/lib64 ]; then
9+
echo /usr/local/nvidia/lib64 > /etc/ld.so.conf.d/nvidia-host.conf
10+
ldconfig
11+
fi
12+
313
if ! command -v nvidia-smi &>/dev/null; then
414
echo "Error: 'nvidia-smi' command not found."
515
exit 1

0 commit comments

Comments
 (0)