This image contains the official Open Data Hub Llama Stack distribution, with all the packages and configuration needed to run a Llama Stack server in a containerized environment.
The image is currently shipping with the Open Data Hub version of Llama Stack version 0.7.1
You can see an overview of the APIs and Providers the image ships with in the table below.
| API | Provider | External? | Enabled by default? | How to enable |
|---|---|---|---|---|
| batches | inline::reference | No | ✅ | N/A |
| files | inline::localfs | No | ✅ | N/A |
| files | remote::s3 | No | ❌ | Set the ENABLE_S3 environment variable |
| inference | inline::sentence-transformers | No | Dependency only* | Requires a custom config.yaml |
| inference | remote::azure | No | ❌ | Set the AZURE_API_KEY environment variable |
| inference | remote::bedrock | No | ❌ | Set the AWS_BEARER_TOKEN_BEDROCK environment variable |
| inference | remote::openai | No | ❌ | Set the OPENAI_API_KEY environment variable |
| inference | remote::vertexai | No | ❌ | Set the VERTEX_AI_PROJECT environment variable |
| inference | remote::vllm | No | ❌ | Set the VLLM_URL environment variable |
| inference | remote::vllm | No | ❌ | Set the VLLM_EMBEDDING_URL environment variable |
| inference | remote::watsonx | No | ❌ | Set the WATSONX_API_KEY environment variable |
| responses | inline::builtin | No | ✅ | N/A |
| tool_runtime | inline::file-search | No | ✅ | N/A |
| tool_runtime | remote::brave-search | No | ✅ | N/A |
| tool_runtime | remote::model-context-protocol | No | ✅ | N/A |
| tool_runtime | remote::tavily-search | No | ✅ | N/A |
| vector_io | inline::faiss | No | Dependency only* | Requires a custom config.yaml |
| vector_io | inline::milvus | No | Dependency only* | Requires a custom config.yaml. Incompatible with multi-worker deployments |
| vector_io | remote::milvus | No | ❌ | Set the MILVUS_ENDPOINT environment variable |
| vector_io | remote::pgvector | No | ❌ | Set the ENABLE_PGVECTOR environment variable |
| vector_io | remote::qdrant | No | ❌ | Set the ENABLE_QDRANT environment variable |
* Dependency only providers are not included in the default runtime config.yaml but their dependencies are pre-installed in the container image. To use them, pass a custom config.yaml at runtime that includes the provider definitions.