Skip to content

Commit 952b9f0

Browse files
feat: add OrcaRouter inference provider
Register a named remote::orcarouter inference provider that mirrors the existing OpenAI-compatible adapters (deepseek, groq, together). The adapter extends OpenAIMixin and only requires get_base_url(), targeting the OrcaRouter aggregation gateway at https://api.orcarouter.ai/v1. - src/ogx/providers/remote/inference/orcarouter/: config + adapter - src/ogx/providers/registry/inference.py: RemoteProviderSpec entry - docs/docs/providers/inference/remote_orcarouter.mdx: generated docs - tests/unit/providers/inference/test_orcarouter_config.py: unit tests Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: XiaoHuo888 <sjh00112233@outlook.com>
1 parent f139a67 commit 952b9f0

7 files changed

Lines changed: 217 additions & 0 deletions

File tree

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
description: "OrcaRouter inference provider for accessing aggregated frontier and open models via the OrcaRouter gateway."
3+
sidebar_label: Remote - Orcarouter
4+
title: remote::orcarouter
5+
---
6+
7+
# remote::orcarouter
8+
9+
## Description
10+
11+
OrcaRouter inference provider for accessing aggregated frontier and open models via the OrcaRouter gateway.
12+
13+
## Configuration
14+
15+
| Field | Type | Required | Default | Description |
16+
|-------|------|----------|---------|-------------|
17+
| `allowed_models` | `list[str] \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
18+
| `refresh_models` | `bool` | No | False | Whether to refresh models periodically from the provider |
19+
| `api_key` | `SecretStr \| None` | No | | Authentication credential for the provider |
20+
| `network` | `NetworkConfig \| None` | No | | Network configuration including TLS, proxy, and timeout settings. |
21+
| `network.tls` | `TLSConfig \| None` | No | | TLS/SSL configuration for secure connections. |
22+
| `network.tls.verify` | `bool \| Path` | No | True | Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file. |
23+
| `network.tls.min_version` | `Literal[TLSv1.2, TLSv1.3] \| None` | No | | Minimum TLS version to use. Defaults to system default if not specified. |
24+
| `network.tls.ciphers` | `list[str] \| None` | No | | List of allowed cipher suites (e.g., ['ECDHE+AESGCM', 'DHE+AESGCM']). |
25+
| `network.tls.client_cert` | `Path \| None` | No | | Path to client certificate file for mTLS authentication. |
26+
| `network.tls.client_key` | `Path \| None` | No | | Path to client private key file for mTLS authentication. |
27+
| `network.proxy` | `ProxyConfig \| None` | No | | Proxy configuration for HTTP connections. |
28+
| `network.proxy.url` | `HttpUrl \| None` | No | | Single proxy URL for all connections (e.g., 'http://proxy.example.com:8080'). |
29+
| `network.proxy.http` | `HttpUrl \| None` | No | | Proxy URL for HTTP connections. |
30+
| `network.proxy.https` | `HttpUrl \| None` | No | | Proxy URL for HTTPS connections. |
31+
| `network.proxy.cacert` | `Path \| None` | No | | Path to CA certificate file for verifying the proxy's certificate. Required for proxies in interception mode. |
32+
| `network.proxy.no_proxy` | `list[str] \| None` | No | | List of hosts that should bypass the proxy (e.g., ['localhost', '127.0.0.1', '.internal.corp']). |
33+
| `network.timeout` | `float \| TimeoutConfig \| None` | No | | Timeout configuration. Can be a float (for both connect and read) or a TimeoutConfig object with separate connect and read timeouts. |
34+
| `network.timeout.connect` | `float \| None` | No | | Connection timeout in seconds. |
35+
| `network.timeout.read` | `float \| None` | No | | Read timeout in seconds. |
36+
| `network.headers` | `dict[str, str] \| None` | No | | Additional HTTP headers to include in all requests. |
37+
| `network.limits` | `LimitsConfig \| None` | No | | HTTP connection pool limits (max connections, keepalive connections, keepalive expiry). None uses httpx's own defaults. |
38+
| `network.limits.max_connections` | `int \| None` | No | 100 | Maximum number of concurrent connections in the pool. None means no limit. Values must be >= 1 if set. |
39+
| `network.limits.max_keepalive_connections` | `int \| None` | No | 20 | Maximum number of idle keep-alive connections to retain. None means no limit. Values must be >= 0 if set. |
40+
| `network.limits.keepalive_expiry` | `float \| None` | No | 5.0 | Time in seconds to keep idle keep-alive connections open before closing them. None means no expiry. Values must be >= 0 if set. |
41+
| `base_url` | `HttpUrl \| None` | No | https://api.orcarouter.ai/v1 | Base URL for the OrcaRouter API |
42+
43+
## Sample Configuration
44+
45+
```yaml
46+
base_url: https://api.orcarouter.ai/v1
47+
api_key: ${env.ORCAROUTER_API_KEY:=}
48+
```

src/ogx/providers/registry/inference.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,16 @@ def available_providers() -> list[ProviderSpec]:
237237
provider_data_validator="ogx.providers.remote.inference.groq.config.GroqProviderDataValidator",
238238
description="Groq inference provider for ultra-fast inference using Groq's LPU technology.",
239239
),
240+
RemoteProviderSpec(
241+
api=Api.inference,
242+
adapter_type="orcarouter",
243+
provider_type="remote::orcarouter",
244+
pip_packages=[],
245+
module="ogx.providers.remote.inference.orcarouter",
246+
config_class="ogx.providers.remote.inference.orcarouter.OrcaRouterImplConfig",
247+
provider_data_validator="ogx.providers.remote.inference.orcarouter.config.OrcaRouterProviderDataValidator",
248+
description="OrcaRouter inference provider for accessing aggregated frontier and open models via the OrcaRouter gateway.",
249+
),
240250
RemoteProviderSpec(
241251
api=Api.inference,
242252
adapter_type="llama-openai-compat",

src/ogx/providers/remote/inference/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ inference/
1919
oci/ # Oracle Cloud Infrastructure GenAI
2020
ollama/ # Ollama (local model serving)
2121
openai/ # OpenAI API
22+
orcarouter/ # OrcaRouter (aggregation gateway)
2223
runpod/ # RunPod cloud GPU
2324
sambanova/ # SambaNova
2425
tgi/ # HuggingFace TGI and Inference API
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
from .config import OrcaRouterImplConfig
8+
9+
10+
async def get_adapter_impl(config: OrcaRouterImplConfig, _deps):
11+
from .orcarouter import OrcaRouterInferenceAdapter
12+
13+
assert isinstance(config, OrcaRouterImplConfig), f"Unexpected config type: {type(config)}"
14+
15+
impl = OrcaRouterInferenceAdapter(config=config)
16+
17+
await impl.initialize()
18+
19+
return impl
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
from typing import Any
8+
9+
from pydantic import BaseModel, Field, HttpUrl, SecretStr
10+
11+
from ogx.providers.utils.inference.model_registry import RemoteInferenceProviderConfig
12+
from ogx_api import json_schema_type
13+
14+
DEFAULT_BASE_URL = "https://api.orcarouter.ai/v1"
15+
16+
17+
class OrcaRouterProviderDataValidator(BaseModel):
18+
"""Validates provider-specific request data for OrcaRouter inference."""
19+
20+
orcarouter_api_key: SecretStr | None = Field(
21+
default=None,
22+
description="API key for OrcaRouter models",
23+
)
24+
25+
26+
@json_schema_type
27+
class OrcaRouterImplConfig(RemoteInferenceProviderConfig):
28+
"""Configuration for the OrcaRouter inference provider."""
29+
30+
base_url: HttpUrl | None = Field(
31+
default=HttpUrl(DEFAULT_BASE_URL),
32+
description="Base URL for the OrcaRouter API",
33+
)
34+
35+
@classmethod
36+
def sample_run_config(cls, api_key: str = "${env.ORCAROUTER_API_KEY:=}", **kwargs) -> dict[str, Any]:
37+
return {
38+
"base_url": DEFAULT_BASE_URL,
39+
"api_key": api_key,
40+
}
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
from collections.abc import AsyncIterator
8+
9+
from ogx.providers.remote.inference.orcarouter.config import OrcaRouterImplConfig
10+
from ogx.providers.utils.inference.openai_mixin import OpenAIMixin
11+
from ogx_api import OpenAICompletion, OpenAICompletionRequestWithExtraBody
12+
13+
14+
class OrcaRouterInferenceAdapter(OpenAIMixin):
15+
"""Inference adapter for the OrcaRouter gateway platform.
16+
17+
OrcaRouter is an OpenAI-compatible aggregation gateway that routes requests
18+
to a range of frontier and open models through a single endpoint. The shared
19+
`OpenAIMixin` handles chat completions and embeddings once pointed at
20+
OrcaRouter's base URL. See https://www.orcarouter.ai.
21+
"""
22+
23+
config: OrcaRouterImplConfig
24+
25+
provider_data_api_key_field: str = "orcarouter_api_key"
26+
27+
def get_base_url(self) -> str:
28+
return str(self.config.base_url)
29+
30+
async def openai_completion(
31+
self,
32+
params: OpenAICompletionRequestWithExtraBody,
33+
) -> OpenAICompletion | AsyncIterator[OpenAICompletion]:
34+
"""OrcaRouter does not support the legacy /v1/completions endpoint.
35+
36+
OrcaRouter is a chat-first aggregation gateway, so the legacy
37+
OpenAI completions endpoint is not part of its API surface.
38+
"""
39+
raise NotImplementedError(
40+
"OrcaRouter does not support /v1/completions endpoint. Only /v1/chat/completions is supported."
41+
)
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
import pytest
8+
9+
from ogx.core.stack import replace_env_vars
10+
from ogx.providers.remote.inference.orcarouter.config import OrcaRouterImplConfig
11+
from ogx.providers.remote.inference.orcarouter.orcarouter import OrcaRouterInferenceAdapter
12+
from ogx_api import OpenAICompletionRequestWithExtraBody
13+
14+
15+
class TestOrcaRouterConfig:
16+
"""Tests for the OrcaRouter inference provider config and adapter wiring."""
17+
18+
def test_default_base_url(self):
19+
config = OrcaRouterImplConfig(api_key="test-key")
20+
adapter = OrcaRouterInferenceAdapter(config=config)
21+
adapter.provider_data_api_key_field = None
22+
23+
assert adapter.get_base_url() == "https://api.orcarouter.ai/v1"
24+
25+
def test_custom_base_url_from_config(self):
26+
custom_url = "https://custom.orcarouter.ai/v1"
27+
config = OrcaRouterImplConfig(api_key="test-key", base_url=custom_url)
28+
adapter = OrcaRouterInferenceAdapter(config=config)
29+
adapter.provider_data_api_key_field = None
30+
31+
assert adapter.get_base_url() == custom_url
32+
33+
def test_sample_run_config_uses_env_placeholder(self):
34+
cfg = OrcaRouterImplConfig.sample_run_config()
35+
assert cfg["base_url"] == "https://api.orcarouter.ai/v1"
36+
assert cfg["api_key"] == "${env.ORCAROUTER_API_KEY:=}"
37+
38+
def test_sample_run_config_env_expansion(self):
39+
config_data = OrcaRouterImplConfig.sample_run_config(api_key="test-key")
40+
processed_config = replace_env_vars(config_data)
41+
config = OrcaRouterImplConfig.model_validate(processed_config)
42+
43+
assert str(config.base_url) == "https://api.orcarouter.ai/v1"
44+
45+
def test_provider_data_api_key_field(self):
46+
config = OrcaRouterImplConfig(api_key="test-key")
47+
adapter = OrcaRouterInferenceAdapter(config=config)
48+
assert adapter.provider_data_api_key_field == "orcarouter_api_key"
49+
50+
async def test_legacy_completions_endpoint_not_supported(self):
51+
"""OrcaRouter does not support the legacy /v1/completions endpoint."""
52+
config = OrcaRouterImplConfig(api_key="test-key")
53+
adapter = OrcaRouterInferenceAdapter(config=config)
54+
55+
params = OpenAICompletionRequestWithExtraBody(model="anthropic/claude-sonnet-4.6", prompt="Hello")
56+
57+
with pytest.raises(NotImplementedError, match="does not support /v1/completions endpoint"):
58+
await adapter.openai_completion(params)

0 commit comments

Comments
 (0)