Skip to content

PROVIDERS.md

Latest commit

 

History

History
164 lines (120 loc) · 9.94 KB

PROVIDERS.md

File metadata and controls

164 lines (120 loc) · 9.94 KB
Raw

Provider Configuration Guide

relay-ai uses a Native Provider Registry to store configuration and API keys securely in your OS keychain. This guide outlines all available providers, what they do, and common gotchas (like multiple variants of the same provider).

Native Providers

When you run relay-ai providers add, you can select from the following templates. The CLI automatically configures the correct endpoint format (@ai-sdk/openai-compatible vs specific SDKs) and fetches available models.

Anthropic

  • Description: The official Anthropic API for Claude models.
  • Base URL: https://api.anthropic.com
  • Known Issues: None. Highly recommended for standard Claude access.

OpenAI

  • Description: The official OpenAI API for GPT models, o-series reasoning models, and Codex.
  • Base URL: https://api.openai.com/v1

Google Gemini

  • Description: The official Google Generative Language API for Gemini models.
  • Base URL: https://generativelanguage.googleapis.com/v1beta/openai

Groq

  • Description: Ultra-fast inference API hosting open-weight models (Llama, Mixtral).
  • Base URL: https://api.groq.com/openai/v1

Mistral

  • Description: Official API for Mistral models.
  • Base URL: https://api.mistral.ai/v1
  • Gotchas / Known Issues: Mistral's free tier has strict API rate limits (HTTP 429). Tool-heavy coding sessions can burn through your quota very quickly due to parallel requests.

Together AI

  • Description: Platform for training, fine-tuning, and running open-source models.
  • Base URL: https://api.together.xyz/v1

Cerebras

  • Description: High-speed AI inference platform powered by wafer-scale chips.
  • Base URL: https://api.cerebras.ai/v1

DeepInfra

  • Description: Serverless inference for top open-source models.
  • Base URL: https://api.deepinfra.com/v1/openai

DeepSeek

  • Description: API for DeepSeek's coding and chat models.
  • Base URL: https://api.deepseek.com/v1

Zhipu AI (GLM)

  • Description: Chinese provider hosting the GLM model family.
  • Base URL: https://open.bigmodel.cn/api/paas/v4

Qwen Cloud and Alibaba DashScope

Relay offers three separate API-key provider entries for Qwen/DashScope. They use distinct accounts, endpoints, billing, and API keys: a key from one entry is not interchangeable with another. These are API keys, not OAuth sign-ins.

1. Alibaba DashScope (China)

  • Base URL: https://dashscope.aliyuncs.com/compatible-mode/v1
  • Description: The existing mainland China DashScope platform for Qwen and other Alibaba models.

2. Qwen Cloud (Pay-As-You-Go)

  • Base URL: https://dashscope-intl.aliyuncs.com/compatible-mode/v1
  • API keys: Create a Qwen Cloud API key at https://home.qwencloud.com/api-keys.
  • Billing: Standard pay-as-you-go pricing. Relay can show Alibaba pricing metadata for this variant.

3. Qwen Cloud (Token Plan)

  • Base URL: https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
  • API keys: Create a Qwen Cloud API key at https://home.qwencloud.com/api-keys.
  • Billing: Uses Token Plan credits, not pay-as-you-go rates. Relay intentionally does not show Alibaba PAYG prices for this option because they would be misleading.

Relay restricts these entries to models suitable for interactive coding and agent tool use, so image/audio-only and no-tool catalog rows are not offered. Token Plan also remains available through Relay's API Server because it is commonly used as a coding/agent bridge.

Cloudflare Workers AI

  • Description: Cloudflare's hosted inference platform for open-weight models (Llama, Gemma, GLM, Kimi, and others).
  • Base URL: https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/v1
  • Setup: You need both a Cloudflare Account ID (from the Cloudflare dashboard) and an API token, not just an API key — the CLI prompts for the Account ID separately and substitutes it into the base URL.
  • Gotchas / Known Issues:
    • Standard models get 10,000 free Neurons/day; this is a real rate limit and tool-heavy coding sessions (large system prompts, many MCP tools) can hit HTTP 429 Too Many Requests quickly, especially on the free tier.
    • Many Cloudflare models have small context windows (24K–131K). Relay hides models below 128K from Claude Code, Codex, and Gemini (136K for Antigravity, which enforces a stricter internal minimum) since agent system prompts and tool definitions alone can consume 25K+ tokens.

xAI

  • Description: API for Grok models.
  • Base URL: https://api.x.ai/v1

Perplexity

  • Description: API for Perplexity's online models (Sonar).
  • Base URL: https://api.perplexity.ai

Cohere

  • Description: API for Command models.
  • Base URL: https://api.cohere.com/compatibility/v1

OpenRouter

  • Description: Unified API proxy providing access to dozens of different models.
  • Base URL: https://openrouter.ai/api/v1

ClinePass

  • Description: Cline's paid coding model subscription, available through either an API key or Cline account OAuth.
  • Unified CLI setup: Run relay-ai providers, choose + Add a provider → ClinePass, then choose Use an API key or Sign in with ClinePass. For an existing ClinePass entry, choose Change authentication (API/OAuth) to switch methods.
  • API key setup: Get a key from the Cline account settings, then paste it into the CLI or UI. Relay validates the key before saving it.
  • OAuth setup: Run relay-ai providers auth cline-pass, or choose Sign in with ClinePass in the unified CLI flow or UI.
  • Usage and billing: Both authentication methods use the ClinePass account and its subscription limits. Relay does not create a separate pay-as-you-go account or bill.
  • Endpoint: https://api.cline.bot/api/v1
  • Model IDs: Relay refreshes ClinePass's live catalog and preserves full IDs such as cline-pass/qwen3.8-max.
  • Credential isolation: API keys and OAuth tokens use separate keychain entries. Switching methods replaces the old credential only after the new one is saved.
  • Troubleshooting: If validation fails, the wizard shows the provider's error and confirms that no registry entry was saved. Add --trace to write a redacted diagnostic log under ~/.relay-ai/logs/.

Local Models (Ollama & LM Studio)

  • Description: Connects to locally running inference engines.
  • Base URLs: Custom prompts ask for your local URL (e.g., http://127.0.0.1:11434/v1).
  • Gotchas: You can skip providing an API key since local APIs generally don't require auth.

Subscription Providers (OAuth)

Relay AI can connect supported subscriptions with a one-time device code instead of an API key:

  • GitHub Copilot: relay-ai providers auth github-copilot
  • OpenAI ChatGPT: relay-ai providers auth openai-oauth
  • xAI SuperGrok: relay-ai providers auth xai-oauth
  • ClinePass: relay-ai providers auth cline-pass

You can also connect them from Providers & Keys in relay-ai ui. Relay AI displays the device code, provides a Copy code button, and opens the provider's sign-in page only after you select Open sign-in page.

GitHub Copilot catalogs are plan-aware. Paid accounts receive the callable chat models returned for that account. Free accounts receive only the verified Free-compatible models. If the plan cannot be verified, Relay AI uses the same conservative Free policy so it does not expose models that may consume paid requests. Use Refresh Models after changing plans.

ClinePass supports both API-key and OAuth authentication. OAuth access tokens are stored without a transport prefix and are formatted as workos: only for ClinePass requests. The runtime retries one expired-token request after refreshing OAuth; it does not loop indefinitely.

See Subscription OAuth for complete setup and troubleshooting instructions.

The Moonshot / Kimi Confusion

Moonshot AI has split their product into three separate platforms. They all share the "Kimi" name, but they use different billing systems, different base URLs, and their API keys are not interchangeable.

If you use the wrong provider template for your key, you will receive a 401 Invalid Authentication or Incorrect API key error.

1. Moonshot (Kimi)

  • Platform: Chinese Domestic Developer Platform (platform.moonshot.cn)
  • Base URL: https://api.moonshot.cn/v1
  • Description: The standard, pay-as-you-go developer platform for users in China. It provides access to the standard moonshot-v1 models.

2. Moonshot Global (kimi.ai)

  • Platform: Global Developer Platform (platform.kimi.ai)
  • Base URL: https://api.moonshot.ai/v1
  • Description: The standard, pay-as-you-go developer platform for international users. Crucially, the global platform also grants access to kimi-k2.7-code directly through the standard API, bypassing the need for a separate coding subscription. If you generated a key at platform.kimi.ai, use this provider!

3. Kimi Code (Subscription Required)

  • Platform: Kimi Code Subscription Console (www.kimi.com/code/console)
  • Base URL: https://api.kimi.com/coding/v1
  • Description: A standalone monthly subscription product specifically geared towards coding agents. If you bought a monthly "Kimi Plus" or "Kimi Code" subscription, you must generate your API key directly from the coding console, and select this provider in relay-ai.

Unsupported / Advanced Providers

  • Amazon Bedrock: Not supported as a first-class API-key provider yet (needs AWS credentials). If you already configured Bedrock in OpenCode CLI, you can optionally run relay-ai providers import.
  • Azure OpenAI: Not supported as a first-class provider yet (needs per-model deployment URLs). Optional: relay-ai providers import if already set up in OpenCode CLI.
  • Google Vertex AI: Use relay-ai server --vertex with gcloud Application Default Credentials. No API key in the registry.