feat: add vLLM LXC container script by piotrlaczykowski · Pull Request #1678 · community-scripts/ProxmoxVED

piotrlaczykowski · 2026-04-07T19:19:54Z

✍️ Description

Adds `ct/vllm.sh` and `install/vllm-install.sh` for a privileged Ubuntu 24.04 LXC running vLLM, an OpenAI-compatible inference server for local LLM inference with NVIDIA GPU passthrough.

Key design decisions:

Privileged container (`var_unprivileged=0`) — required for NVIDIA GPU passthrough on Proxmox
Ubuntu 24.04 — required for CUDA 12.x library compatibility
Python venv at `/opt/vllm` — isolated from system Python
`nvidia-smi` used for GPU detection (more reliable than `nvcc` post-driver-install)
Systemd service uses a wrapper script `/usr/local/bin/vllm-server` that sources `/etc/vllm/vllm.env` — avoids fragile heredoc escaping in ExecStart
Service does not auto-start on first install — user configures `/etc/vllm/vllm.env` first, then runs `vllm-cli start`
Default model: `Qwen/Qwen2.5-1.5B-Instruct` (no Hugging Face gate, small RAM footprint)
`vllm-cli` helper: `start|stop|restart|status|logs|config|models|version`
Update script follows `ollama.sh` version-check pattern; version tracked in `/opt/vLLM_version.txt`

Default resources: 8 vCPU · 16 GB RAM · 40 GB disk · Ubuntu 24.04

🔗 Related PR / Issue

Link: N/A — new script submission

✅ Prerequisites (X in brackets)

Self-review completed – Code follows project standards.
Tested thoroughly – Changes work as expected.
No breaking changes – Existing functionality remains intact.
No security risks – No hardcoded secrets, unnecessary privilege escalations, or permission issues.

🛠️ Type of Change (X in brackets)

🐞 Bug fix – Resolves an issue without breaking functionality.
✨ New feature – Adds new, non-breaking functionality.
💥 Breaking change – Alters existing functionality in a way that may require updates.
🆕 New script – A fully functional and tested script or script set.
🌍 Website update – Changes to website-related JSON files or metadata.
🔧 Refactoring / Code Cleanup – Improves readability or maintainability without changing functionality.
📝 Documentation update – Changes to `README`, `AppName.md`, `CONTRIBUTING.md`, or other docs.

🔍 Code & Security Review (X in brackets)

Follows `Code_Audit.md` & `CONTRIBUTING.md` guidelines
Uses correct script structure (`AppName.sh`, `AppName-install.sh`)
No hardcoded credentials

📋 Additional Information (optional)

The service intentionally does not auto-start on first install. GPU model loading can take several minutes depending on model size and hardware. Users should edit `/etc/vllm/vllm.env` to set their preferred model, then start with `vllm-cli start`.

📦 Application Requirements (for new scripts)

The application is at least 6 months old — first release Feb 2023
The application is actively maintained — last commit today (Apr 7 2026)
The application has 600+ GitHub stars — 75,591 stars
Official release tarballs are published — https://github.com/vllm-project/vllm/releases
I understand that not all scripts will be accepted due to various reasons and criteria by the community-scripts ORG

🌐 Source

Project: https://github.com/vllm-project/vllm
Docs: https://docs.vllm.ai

Adds privileged Ubuntu 24.04 LXC container for vLLM, an OpenAI-compatible inference server supporting NVIDIA GPU passthrough with CPU fallback. - Privileged container (var_unprivileged=0) for GPU passthrough - Ubuntu 24.04 for CUDA compatibility - Python venv install with nvidia-smi GPU detection and CPU fallback - Clean wrapper script pattern for systemd service (avoids heredoc escaping) - vllm-cli helper for start/stop/logs/config/models/version - Default model: Qwen/Qwen2.5-1.5B-Instruct (gated-free, small footprint) - OpenAI-compatible API on port 8000 Co-Authored-By: claude-flow <ruv@ruv.net>

CrazyWolf13 · 2026-04-07T19:30:56Z

Why was the PR template removed?

piotrlaczykowski · 2026-04-07T19:37:17Z

Apologies for the missing template — updated the PR description to fill in the full ProxmoxVED template including all prerequisites, code/security review checklist, and application requirements (vLLM: 75k+ stars, active since Feb 2023, releases published).

CrazyWolf13 · 2026-04-07T19:54:48Z

we normally prefer uv based venv to not interfere with system packages (this got suggested by some people)

Also you seem to be missing the json file for the frontend, can you add that?

check json folder and docs folder for help.

piotrlaczykowski · 2026-04-15T21:58:50Z

we normally prefer uv based venv to not interfere with system packages (this got suggested by some people)

Also you seem to be missing the json file for the frontend, can you add that?

check json folder and docs folder for help.

Fixed :)

… + add model requirements note

github-actions · 2026-04-28T09:31:11Z

@piotrlaczykowski This PR has been marked as stale. It will be closed if no new commits are added in 7 days.

- ct/vllm.sh: use check_for_gh_release + CHECK_UPDATE_RELEASE instead of manual get_latest_gh_tag + version file comparison - ct/vllm.sh: lower default resources to 4 CPU / 8192 MB RAM - install/vllm-install.sh: use get_latest_github_release (strips v prefix) instead of get_latest_gh_tag + manual ${RELEASE#v} stripping - install/vllm-install.sh: default model changed to Qwen/Qwen2.5-0.5B-Instruct (smallest ungated model), add comments with examples for other sizes - json/vllm.json: update install_methods resources to 4 CPU / 8192 MB RAM - json/vllm.json: update warning note to reflect new resource defaults

piotrlaczykowski · 2026-04-28T18:26:34Z

All resolved

github-actions · 2026-04-29T02:14:48Z

@piotrlaczykowski Recent activity detected. Removing stale label.

piotrlaczykowski · 2026-05-12T21:16:05Z

Anyone to review? @CrazyWolf13

CrazyWolf13 · 2026-05-13T05:24:53Z

@piotrlaczykowski then request a review ^^

CrazyWolf13 · 2026-05-13T05:25:36Z

+
+  if check_for_gh_release "vLLM" "vllm-project/vllm"; then
+    RELEASE="${CHECK_UPDATE_RELEASE}"
+    RELEASE_VERSION="${RELEASE#v}"


I think we don't need this extra var and can apply ${RELEASE#v} directly where needed

CrazyWolf13 · 2026-05-13T05:25:51Z

+
+    msg_info "Updating ${APP} to ${RELEASE}"
+    $STD uv pip install --python /opt/vllm/.venv/bin/python --upgrade "vllm==${RELEASE_VERSION}"
+    msg_ok "Updated ${APP} to ${RELEASE}"


can you directly resovle the $app var

CrazyWolf13 · 2026-05-13T05:26:31Z

+echo -e "${INFO}${YW} Swagger docs:${CL}"
+echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:8000/docs${CL}"


those probably don't need an extra info, as /docs is pretty much the standard and the api probably also not.

CrazyWolf13 · 2026-05-13T05:26:50Z

+
+RELEASE=$(get_latest_github_release "vllm-project/vllm")
+
+msg_info "Installing ${APP} v${RELEASE} (Patience — this takes 5-15 minutes)"


resolve $app

CrazyWolf13 · 2026-05-13T05:27:43Z

+  msg_info "No GPU detected — installing vLLM with CPU/OpenVINO backend"
+  $STD uv pip install --python /opt/vllm/.venv/bin/python "vllm==${RELEASE}" --extra-index-url https://download.pytorch.org/whl/cpu
+fi
+echo "v${RELEASE}" >/opt/vLLM_version.txt


can you do this to ~/.vllm that is where we normally put it.

CrazyWolf13 · 2026-05-13T05:28:26Z

+msg_info "Configuring ${APP}"
+mkdir -p /etc/vllm /opt/vllm-models
+cat <<EOF >/etc/vllm/vllm.env
+# Set MODEL to a Hugging Face model ID before starting the service.
+# Example (small, ungated): Qwen/Qwen2.5-0.5B-Instruct
+# Example (medium):        Qwen/Qwen2.5-7B-Instruct
+# Example (gated):         meta-llama/Llama-3.1-8B-Instruct (requires HF_TOKEN)
+MODEL="Qwen/Qwen2.5-0.5B-Instruct"
+HOST="0.0.0.0"
+PORT="8000"
+GPU_MEM_UTIL="0.90"
+MAX_MODEL_LEN="4096"
+TENSOR_PARALLEL_SIZE="1"
+QUANTIZATION=""
+# HF_TOKEN=""
+HF_HOME="/opt/vllm-models"
+EOF
+msg_ok "Configured ${APP}"
+
+msg_info "Creating vLLM Server Wrapper"


Suggested change

msg_info "Configuring ${APP}"

mkdir -p /etc/vllm /opt/vllm-models

cat <<EOF >/etc/vllm/vllm.env

# Set MODEL to a Hugging Face model ID before starting the service.

# Example (small, ungated): Qwen/Qwen2.5-0.5B-Instruct

# Example (medium): Qwen/Qwen2.5-7B-Instruct

# Example (gated): meta-llama/Llama-3.1-8B-Instruct (requires HF_TOKEN)

MODEL="Qwen/Qwen2.5-0.5B-Instruct"

HOST="0.0.0.0"

PORT="8000"

GPU_MEM_UTIL="0.90"

MAX_MODEL_LEN="4096"

TENSOR_PARALLEL_SIZE="1"

QUANTIZATION=""

# HF_TOKEN=""

HF_HOME="/opt/vllm-models"

EOF

msg_ok "Configured ${APP}"

msg_info "Creating vLLM Server Wrapper"

msg_info "Configuring ${APP}"

mkdir -p /etc/vllm /opt/vllm-models

cat <<EOF >/etc/vllm/vllm.env

# Set MODEL to a Hugging Face model ID before starting the service.

# Example (small, ungated): Qwen/Qwen2.5-0.5B-Instruct

# Example (medium): Qwen/Qwen2.5-7B-Instruct

# Example (gated): meta-llama/Llama-3.1-8B-Instruct (requires HF_TOKEN)

MODEL="Qwen/Qwen2.5-0.5B-Instruct"

HOST="0.0.0.0"

PORT="8000"

GPU_MEM_UTIL="0.90"

MAX_MODEL_LEN="4096"

TENSOR_PARALLEL_SIZE="1"

QUANTIZATION=""

# HF_TOKEN=""

HF_HOME="/opt/vllm-models"

EOF

No extra msg block needed for such short actions

CrazyWolf13 · 2026-05-13T05:29:23Z

+  "interface_port": 8000,
+  "documentation": "https://docs.vllm.ai",
+  "website": "https://github.com/vllm-project/vllm",
+  "logo": "https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/logos/vllm-logo-text-light.png",


utilize one from selfh.st/icons

CrazyWolf13 · 2026-05-13T05:29:51Z

+        "cpu": 4,
+        "ram": 8192,
+        "hdd": 40,
+        "os": "Ubuntu",


is ubuntu preferrable from debian in any way?
because of drivers?

CrazyWolf13 · 2026-05-13T05:30:50Z

+  },
+  "notes": [
+    {
+      "text": "GPU passthrough requires a privileged LXC container with NVIDIA drivers installed on the Proxmox host. Edit /etc/vllm/vllm.env to configure model and settings before starting the service.",


that is just wrong, it does not need to be priviledged for gpu passthrough

MickLesk · 2026-06-01T06:19:32Z

no feedback, stale pr / closed

piotrlaczykowski requested a review from a team as a code owner April 7, 2026 19:19

github-actions Bot added the new script label Apr 7, 2026

fix(vllm): switch to uv-based venv, add json frontend file

b404b44

CrazyWolf13 requested changes Apr 16, 2026

View reviewed changes

Comment thread ct/vllm.sh Outdated

Comment thread install/vllm-install.sh Outdated

Comment thread json/vllm.json Outdated

fix(vllm): use get_latest_gh_tag, lower default resources to 2CPU/4GB…

ad9a7d7

… + add model requirements note

piotrlaczykowski requested a review from CrazyWolf13 April 21, 2026 09:51

CrazyWolf13 requested changes Apr 21, 2026

View reviewed changes

Comment thread ct/vllm.sh Outdated

Comment thread install/vllm-install.sh Outdated

Comment thread install/vllm-install.sh Outdated

MickLesk added the stale label Apr 28, 2026

github-actions Bot removed the stale label Apr 29, 2026

CrazyWolf13 reviewed May 13, 2026

View reviewed changes

CrazyWolf13 requested changes May 13, 2026

View reviewed changes

github-actions Bot added the stale label May 28, 2026

MickLesk closed this Jun 1, 2026

		echo -e "${INFO}${YW} Swagger docs:${CL}"
		echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:8000/docs${CL}"


		RELEASE=$(get_latest_github_release "vllm-project/vllm")

		msg_info "Installing ${APP} v${RELEASE} (Patience — this takes 5-15 minutes)"

Uh oh!

Uh oh!

Conversation

piotrlaczykowski commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✍️ Description

🔗 Related PR / Issue

✅ Prerequisites (X in brackets)

🛠️ Type of Change (X in brackets)

🔍 Code & Security Review (X in brackets)

📋 Additional Information (optional)

📦 Application Requirements (for new scripts)

🌐 Source

Uh oh!

CrazyWolf13 commented Apr 7, 2026

Uh oh!

piotrlaczykowski commented Apr 7, 2026

Uh oh!

CrazyWolf13 commented Apr 7, 2026

Uh oh!

piotrlaczykowski commented Apr 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Apr 28, 2026

Uh oh!

piotrlaczykowski commented Apr 28, 2026

Uh oh!

github-actions Bot commented Apr 29, 2026

Uh oh!

piotrlaczykowski commented May 12, 2026

Uh oh!

CrazyWolf13 commented May 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MickLesk commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

piotrlaczykowski commented Apr 7, 2026 •

edited

Loading