fix(ci): pin validated GPU vLLM runtime

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

Author: skamenan7

.github/actions/setup-vllm-gpu/action.yml

@@ -22,6 +22,10 @@ inputs:
     description: 'Optional vLLM quantization method (awq, gptq, or none)'
     required: false
     default: 'none'
+  vllm-version:
+    description: 'vLLM version to install, or latest for the newest available release'
+    required: false
+    default: '0.22.1'
 
 outputs:
   vllm-url:
@@ -51,7 +55,9 @@ runs:
       shell: bash
       run: |
         if [ -z "${CUDA_HOME:-}" ]; then
-          if [ -d /usr/local/cuda ]; then
+          if [ -d /usr/local/cuda-13.0 ]; then
+            export CUDA_HOME=/usr/local/cuda-13.0
+          elif [ -d /usr/local/cuda ]; then
             export CUDA_HOME=/usr/local/cuda
           elif [ -d /usr/local/cuda-12.8 ]; then
             export CUDA_HOME=/usr/local/cuda-12.8
@@ -83,11 +89,17 @@ runs:
 
     - name: Install vLLM with GPU support
       shell: bash
+      env:
+        VLLM_VERSION: ${{ inputs.vllm-version }}
       run: |
         source /tmp/vllm-env/bin/activate
 
         echo "=== Installing vLLM with CUDA support ==="
-        uv pip install vllm --torch-backend=auto
+        if [ "$VLLM_VERSION" = "latest" ]; then
+          uv pip install vllm --torch-backend=auto
+        else
+          uv pip install "vllm==$VLLM_VERSION" --torch-backend=auto
+        fi
 
         echo "=== Verifying installation ==="
         python -c "import torch; print(f'PyTorch version: {torch.__version__}')"

.github/workflows/launch-gpu-ec2-runner.yml

@@ -114,6 +114,7 @@ jobs:
           gpu-memory-utilization: '0.85'
           max-model-len: '8192'
           quantization: 'none'
+          vllm-version: '0.22.1'
 
       - name: Setup test environment
         uses: ./.github/actions/setup-test-environment

.github/workflows/record-vllm-gpu-tests.yml

@@ -114,6 +114,7 @@ jobs:
           gpu-memory-utilization: '0.85'
           max-model-len: '8192'
           quantization: 'none'
+          vllm-version: '0.22.1'
 
       - name: Setup test environment
         uses: ./.github/actions/setup-test-environment

AWS_SETUP_GUIDE.md

@@ -303,14 +303,14 @@ sudo apt-get install -y \
 sudo apt-get install -y ubuntu-drivers-common
 sudo ubuntu-drivers autoinstall
 
-# Install CUDA 12.8
+# Install CUDA 13.0
 wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
 sudo dpkg -i cuda-keyring_1.1-1_all.deb
 sudo apt-get update
-sudo apt-get -y install cuda-12-8
+sudo apt-get -y install cuda-13-0
 
 # Configure CUDA environment
-echo 'export CUDA_HOME=/usr/local/cuda-12.8' | sudo tee -a /etc/profile.d/cuda.sh
+echo 'export CUDA_HOME=/usr/local/cuda-13.0' | sudo tee -a /etc/profile.d/cuda.sh
 echo 'export PATH=$PATH:$CUDA_HOME/bin' | sudo tee -a /etc/profile.d/cuda.sh
 echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_HOME/lib64' | sudo tee -a /etc/profile.d/cuda.sh
 
@@ -338,7 +338,7 @@ sudo apt-get install -y python3.12 python3.12-venv python3.12-dev
 # Verify installation
 nvidia-smi
 nvcc --version
-docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi
+docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu22.04 nvidia-smi
 
 echo "Setup complete! Ready to create AMI."
 ```
@@ -352,8 +352,8 @@ After setup completes:
 aws ec2 create-image \
   --region us-east-2 \
   --instance-id i-xxxxx \
-  --name "llama-stack-gpu-ubuntu-2204-cuda-12.8-$(date +%Y%m%d)" \
-  --description "Ubuntu 22.04 with NVIDIA drivers, CUDA 12.8, Docker, Python 3.12" \
+  --name "llama-stack-gpu-ubuntu-2204-cuda-13.0-$(date +%Y%m%d)" \
+  --description "Ubuntu 22.04 with NVIDIA drivers, CUDA 13.0, Docker, Python 3.12" \
   --tag-specifications 'ResourceType=image,Tags=[{Key=Name,Value=llama-stack-gpu-ami}]'
 
 ```
@@ -374,7 +374,7 @@ aws ec2 run-instances \
 ssh ubuntu@<ip>
 nvidia-smi
 nvcc --version
-docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi
+docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu22.04 nvidia-smi
 ```
 
 ## Step 4: Create GitHub Personal Access Token

IMPLEMENTATION_PLAN.md

@@ -66,7 +66,7 @@ RELEASE_PAT=ghp_xxxxx (GitHub PAT with 'repo' scope)
 **Actions**:
 
 - [ ] Launch base EC2 instance (g6.2xlarge with Amazon Linux 2023 or Ubuntu 22.04)
-- [ ] Install NVIDIA drivers and CUDA 12.8
+- [ ] Install NVIDIA drivers and CUDA 13.0
 - [ ] Install Docker with NVIDIA Container Toolkit
 - [ ] Install system packages (gcc, g++, make, git, python3.12, python3.12-devel)
 - [ ] Configure CUDA environment variables
@@ -80,8 +80,8 @@ RELEASE_PAT=ghp_xxxxx (GitHub PAT with 'repo' scope)
 
 ```bash
 nvidia-smi  # Should show GPU
-nvcc --version  # Should show CUDA 12.8
-docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi
+nvcc --version  # Should show CUDA 13.0
+docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu22.04 nvidia-smi
 ```
 
 **Dependencies**: Task #11
@@ -126,6 +126,7 @@ docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi
 **Key Features**:
 
 - Install vLLM with GPU support
+- Pin vLLM to 0.22.1 for the validated CUDA 13.0 AMI path
 - Pull gpt-oss:20b model (or specified model)
 - Start vLLM server with optimal settings:
   - Native GPT-OSS MXFP4 weights without an extra vLLM quantization flag

IMPLEMENTATION_STATUS.md

@@ -110,7 +110,8 @@
 
 - [ ] Launch g6.2xlarge instance
 - [ ] Install NVIDIA drivers
-- [ ] Install CUDA 12.8
+- [ ] Install CUDA 13.0
+- [ ] Install/validate vLLM 0.22.1
 - [ ] Install Docker with NVIDIA Container Toolkit
 - [ ] Install Python 3.12
 - [ ] Create AMI

docs/gpu-runners.md

@@ -166,7 +166,8 @@ Create an AMI in `us-east-2` with:
 
 - Base OS: Amazon Linux 2023 or Ubuntu 22.04
 - NVIDIA drivers
-- CUDA 12.8 runtime
+- CUDA 13.0 runtime
+- vLLM 0.22.1
 - Docker with NVIDIA Container Toolkit
 - Python 3.12