Add Helm values for mounting ConfigMap scripts and customizing vLLM entrypoint arg

[aslom]

Problem

Currently, injecting scripts that run before vLLM (e.g., preflight diagnostics, environment validation) requires manually patching the model server deployment with a multi-operation JSON patch that:

1. Adds a ConfigMap volume
2. Adds a volumeMount to the container
3. Adds environment variables
4. Replaces command from ["vllm", "serve"] to ["bash", "-c"]
5. Replaces args to chain the script before vllm serve via &&

See the full procedure: llm-d-pd-utils SKILL.md – Step 3: Patch the model server deployment

This is error-prone, brittle across chart upgrades, and must be repeated separately for prefill and decode deployments with slightly different port/TP configurations. Users must manually reconstruct the full vllm serve command line (including model name, TP size, GPU memory utilization, etc.) inside the patched args field.

Use Case: Preflight Checks

The llm-d-preflight-checks script runs before vLLM to validate the pod environment (GPU topology, NVLink status, RDMA connectivity, env vars). In pause mode it starts an HTTP server on the vLLM port so K8s probes pass while operators inspect diagnostics or run network tests — then /exit releases the port and vLLM starts normally.

This enables operators to:

• Catch misconfigured GPU topology or missing RDMA devices before vLLM allocates GPU memory
• Run inter-pod network benchmarks (perftest, nixlbench) while pods are up but before serving traffic
• Debug pod startup issues without losing the pod to a CrashLoopBackOff

Without Helm-native support, deploying this requires a 30+ line JSON patch per deployment, which looks like:

kubectl patch deployment optimized-baseline-nvidia-gpu-vllm-decode \
  -n ${NAMESPACE} --type=json -p '[
  {"op": "add", "path": "/spec/template/spec/volumes/-", "value": {"name": "preflight-checks", "configMap": {"name": "llm-d-preflight-checks", "defaultMode": 493}}},
  {"op": "add", "path": "/spec/template/spec/containers/0/volumeMounts/-", "value": {"name": "preflight-checks", "mountPath": "/preflight"}},
  {"op": "add", "path": "/spec/template/spec/containers/0/env", "value": [{"name": "LLMD_PREFLIGHT_CHECKS", "value": "pause"}]},
  {"op": "replace", "path": "/spec/template/spec/containers/0/command", "value": ["bash", "-c"]},
  {"op": "replace", "path": "/spec/template/spec/containers/0/args", "value": ["python3 /preflight/llm-d-preflight-checks.py && vllm serve Qwen/Qwen3-32B --tensor-parallel-size=2 --gpu-memory-utilization=0.95"]}
]'

This must be done separately for each deployment (prefill and decode) with different vLLM args each time.

Proposed Helm Values

modelServer:
  # Mount arbitrary ConfigMaps as volumes (scripts, configs, etc.)
  extraConfigMapVolumes:
    - name: preflight-checks
      configMapName: llm-d-preflight-checks
      mountPath: /preflight
      defaultMode: 0755

  # Extra environment variables for the vLLM container
  extraEnv:
    - name: LLMD_PREFLIGHT_CHECKS
      value: "pause"

  # Commands to run before vLLM starts (chained with &&)
  preStartCommands:
    - "python3 /preflight/llm-d-preflight-checks.py"

With preStartCommands, the chart template would automatically:

• Change command to ["bash", "-c"]
• Prepend the listed commands (joined with &&) before the existing vllm serve ... invocation
• Preserve the existing vLLM args (model, --tensor-parallel-size, --port, etc.) without users needing to restate them

Benefits

Today (manual patch)	With Helm values
30+ line JSON patch per deployment	5-line values override
Must manually reconstruct vllm serve args	Chart preserves existing vLLM args
Breaks on chart upgrades that change container spec	Stable across upgrades
Separate patch for prefill vs decode	Single values file, chart handles both
No visibility in helm diff	Clean diff in values

Additional Consideration: Extra vLLM Args

A related but simpler need: allowing users to pass extra vllm serve flags (e.g., --enable-chunked-prefill, --kv-transfer-config) without overriding the entire args array:

modelServer:
  extraVllmArgs:
    - "--enable-chunked-prefill"
    - "--kv-transfer-config=/path/to/config.json"

This would cover both the "run something before vLLM" and "pass extra flags to vLLM" patterns cleanly through Helm values.

Drafted with Claude Code assitance.

[aslom]

Superseded by llm-d/llm-d#1580