Name of the Script
vLLM
Script Type
CT (LXC Container)
Does this script support arm64?
arm64 not supported
📋 Script Details
vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs, exposing an OpenAI-compatible HTTP API (/v1) on port 8000 so existing OpenAI clients can serve local models. Installed via uv into a venv at /opt/vllm; the target model is set via VLLM_MODEL in /opt/vllm/vllm.env and downloaded from HuggingFace on first start. NVIDIA GPU passthrough is recommended for usable performance (vllm was added to the GPU passthrough app list; CPU-only inference is impractically slow).
Name of the Script
vLLM
Script Type
CT (LXC Container)
Does this script support arm64?
arm64 not supported
📋 Script Details
vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs, exposing an OpenAI-compatible HTTP API (/v1) on port 8000 so existing OpenAI clients can serve local models. Installed via uv into a venv at /opt/vllm; the target model is set via VLLM_MODEL in /opt/vllm/vllm.env and downloaded from HuggingFace on first start. NVIDIA GPU passthrough is recommended for usable performance (vllm was added to the GPU passthrough app list; CPU-only inference is impractically slow).