| id | supported-providers |
|---|---|
| title | Supported AI Providers |
| sidebar_position | 8 |
Since the Envoy AI Gateway is designed to provide a Unified API for routing and managing LLM/AI traffic, it supports various AI providers out of the box.
A "support of provider" means two things: the API schema support and the Authentication support.
The former can be configured in the AIServiceBackend resource's schema field, while the latter is configured in the BackendSecurityPolicy.
Below is a table of currently supported providers and their respective configurations.
| Provider Name | API Schema Config on AIServiceBackend | Upstream Authentication Config on BackendSecurityPolicy | Status | Note |
|---|---|---|---|---|
| OpenAI | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| AWS Bedrock | {"name":"AWSBedrock"} |
AWS Bedrock Credentials | ✅ | |
| Azure OpenAI | {"name":"AzureOpenAI","version":"2025-01-01-preview"} or {"name":"OpenAI", "prefix": "/openai/v1"} |
Azure Credentials or Azure API Key | ✅ | |
| Google Gemini on AI Studio | {"name":"OpenAI","prefix":"/v1beta/openai"} |
API Key | ✅ | Only the OpenAI compatible endpoint |
| Google Vertex AI | {"name":"GCPVertexAI"} |
GCP Credentials | ✅ | Supports ADC, Service Account Keys, and Workload Identity Federation |
| Anthropic on GCP Vertex AI | {"name":"GCPAnthropic", "version":"vertex-2023-10-16"} |
GCP Credentials | ✅ | Native Anthropic and OpenAI endpoints. Supports ADC, Service Account Keys, and Workload Identity Federation |
| Groq | {"name":"OpenAI","prefix":"/openai/v1"} |
API Key | ✅ | |
| Grok | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Together AI | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Cohere | {"name":"Cohere","version":"v2"} or {"name":"OpenAI","prefix":"/compatibility/v1"} |
API Key | ✅ | Supports native Cohere v2 (e.g., /cohere/v2/rerank) and OpenAI-compatible endpoints. |
| Mistral | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| DeepInfra | {"name":"OpenAI","prefix":"/v1/openai"} |
API Key | ✅ | Only the OpenAI compatible endpoint |
| DeepSeek | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Hunyuan | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Tencent LLM Knowledge Engine | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Tetrate Agent Router Service (TARS) | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| SambaNova | {"name":"OpenAI","prefix":"/v1"} |
API Key | ✅ | |
| Self-hosted-models | {"name":"OpenAI","prefix":"/v1"} |
N/A | Depending on the API schema spoken by self-hosted servers. For example, vLLM speaks the OpenAI format. Also, API Key auth can be configured as well. | |
| Anthropic | {"name":"Anthropic"} |
Anthropic API Key | ✅ | Native Anthropic Messages and OpenAI-compatible Chat Completions endpoints |