Skip to content

Commit 74647be

Browse files
NishchayMahormattf
andauthored
feat(inference): add DeepSeek remote inference provider (#6240)
# What does this PR do? Adds a remote inference provider for **DeepSeek** (`remote::deepseek`). DeepSeek is a widely used frontier lab (DeepSeek-V3 chat and DeepSeek-R1 reasoning models) that wasn't yet in the inference lineup — this lets users run those models through the standard inference API. DeepSeek exposes an OpenAI-compatible chat-completions API, so the adapter extends the shared `OpenAIMixin` pointed at `https://api.deepseek.com/v1`, authenticated via `DEEPSEEK_API_KEY` (or the `deepseek_api_key` provider-data header). No new pip dependency is required. DeepSeek does not offer an embeddings endpoint, so `openai_embeddings` is unsupported (mirrors the Cerebras adapter). ## Changes - `providers/remote/inference/deepseek/` — adapter (`DeepSeekInferenceAdapter`), config (`DeepSeekImplConfig` + provider-data validator), and `get_adapter_impl`. - Registered `remote::deepseek` in `providers/registry/inference.py`. - Unit tests in `tests/unit/providers/inference/test_deepseek_config.py`. - Regenerated provider docs (`docs/.../inference/remote_deepseek.mdx`) via `scripts/provider_codegen.py`. ## Test Plan - `pytest tests/unit/providers/inference/test_deepseek_config.py` → 6 passed (default/custom/env base URL, sample run config, provider-data field, embeddings-unsupported). - Provider resolves from the registry (`remote::deepseek`) and imports cleanly. - `mypy` and `ruff` clean on the new module. Usage: ```yaml providers: inference: - provider_id: deepseek provider_type: remote::deepseek config: api_key: ${env.DEEPSEEK_API_KEY} ``` --------- Signed-off-by: Nishchay Mahor <nishchaymahor@gmail.com> Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu>
1 parent 4001d57 commit 74647be

8 files changed

Lines changed: 262 additions & 2 deletions

File tree

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
description: "DeepSeek inference provider for accessing DeepSeek models via the DeepSeek API."
3+
sidebar_label: Remote - Deepseek
4+
title: remote::deepseek
5+
---
6+
7+
# remote::deepseek
8+
9+
## Description
10+
11+
DeepSeek inference provider for accessing DeepSeek models via the DeepSeek API.
12+
13+
## Configuration
14+
15+
| Field | Type | Required | Default | Description |
16+
|-------|------|----------|---------|-------------|
17+
| `allowed_models` | `list[str] \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
18+
| `refresh_models` | `bool` | No | False | Whether to refresh models periodically from the provider |
19+
| `api_key` | `SecretStr \| None` | No | | Authentication credential for the provider |
20+
| `network` | `NetworkConfig \| None` | No | | Network configuration including TLS, proxy, and timeout settings. |
21+
| `network.tls` | `TLSConfig \| None` | No | | TLS/SSL configuration for secure connections. |
22+
| `network.tls.verify` | `bool \| Path` | No | True | Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file. |
23+
| `network.tls.min_version` | `Literal[TLSv1.2, TLSv1.3] \| None` | No | | Minimum TLS version to use. Defaults to system default if not specified. |
24+
| `network.tls.ciphers` | `list[str] \| None` | No | | List of allowed cipher suites (e.g., ['ECDHE+AESGCM', 'DHE+AESGCM']). |
25+
| `network.tls.client_cert` | `Path \| None` | No | | Path to client certificate file for mTLS authentication. |
26+
| `network.tls.client_key` | `Path \| None` | No | | Path to client private key file for mTLS authentication. |
27+
| `network.proxy` | `ProxyConfig \| None` | No | | Proxy configuration for HTTP connections. |
28+
| `network.proxy.url` | `HttpUrl \| None` | No | | Single proxy URL for all connections (e.g., 'http://proxy.example.com:8080'). |
29+
| `network.proxy.http` | `HttpUrl \| None` | No | | Proxy URL for HTTP connections. |
30+
| `network.proxy.https` | `HttpUrl \| None` | No | | Proxy URL for HTTPS connections. |
31+
| `network.proxy.cacert` | `Path \| None` | No | | Path to CA certificate file for verifying the proxy's certificate. Required for proxies in interception mode. |
32+
| `network.proxy.no_proxy` | `list[str] \| None` | No | | List of hosts that should bypass the proxy (e.g., ['localhost', '127.0.0.1', '.internal.corp']). |
33+
| `network.timeout` | `float \| TimeoutConfig \| None` | No | | Timeout configuration. Can be a float (for both connect and read) or a TimeoutConfig object with separate connect and read timeouts. |
34+
| `network.timeout.connect` | `float \| None` | No | | Connection timeout in seconds. |
35+
| `network.timeout.read` | `float \| None` | No | | Read timeout in seconds. |
36+
| `network.headers` | `dict[str, str] \| None` | No | | Additional HTTP headers to include in all requests. |
37+
| `base_url` | `HttpUrl \| None` | No | https://api.deepseek.com/v1 | Base URL for the DeepSeek API |
38+
39+
## Sample Configuration
40+
41+
```yaml
42+
base_url: https://api.deepseek.com/v1
43+
api_key: ${env.DEEPSEEK_API_KEY:=}
44+
```

scripts/generate_target_models_docs.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@
6969
# These inference providers exist in the registry but do not have named
7070
# integration-test setups yet, so they are intentionally excluded from this doc.
7171
INTENTIONALLY_UNMAPPED_REGISTRY_PROVIDERS = {
72+
"deepseek",
7273
"mistral",
7374
"meta",
7475
"nvidia",

src/ogx/providers/registry/inference.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,16 @@ def available_providers() -> list[ProviderSpec]:
7171
provider_data_validator="ogx.providers.remote.inference.mistral.config.MistralProviderDataValidator",
7272
description="Mistral AI inference provider for accessing Mistral models via the Mistral API.",
7373
),
74+
RemoteProviderSpec(
75+
api=Api.inference,
76+
adapter_type="deepseek",
77+
provider_type="remote::deepseek",
78+
pip_packages=[],
79+
module="ogx.providers.remote.inference.deepseek",
80+
config_class="ogx.providers.remote.inference.deepseek.DeepSeekImplConfig",
81+
provider_data_validator="ogx.providers.remote.inference.deepseek.config.DeepSeekProviderDataValidator",
82+
description="DeepSeek inference provider for accessing DeepSeek models via the DeepSeek API.",
83+
),
7484
RemoteProviderSpec(
7585
api=Api.inference,
7686
adapter_type="ollama",
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
from .config import DeepSeekImplConfig
8+
9+
10+
async def get_adapter_impl(config: DeepSeekImplConfig, _deps):
11+
from .deepseek import DeepSeekInferenceAdapter
12+
13+
assert isinstance(config, DeepSeekImplConfig), f"Unexpected config type: {type(config)}"
14+
15+
impl = DeepSeekInferenceAdapter(config=config)
16+
17+
await impl.initialize()
18+
19+
return impl
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
import os
8+
from typing import Any
9+
10+
from pydantic import BaseModel, Field, HttpUrl, SecretStr
11+
12+
from ogx.providers.utils.inference.model_registry import RemoteInferenceProviderConfig
13+
from ogx_api import json_schema_type
14+
15+
DEFAULT_BASE_URL = "https://api.deepseek.com/v1"
16+
17+
18+
class DeepSeekProviderDataValidator(BaseModel):
19+
"""Validates provider-specific request data for DeepSeek inference."""
20+
21+
deepseek_api_key: SecretStr | None = Field(
22+
default=None,
23+
description="API key for DeepSeek models",
24+
)
25+
26+
27+
@json_schema_type
28+
class DeepSeekImplConfig(RemoteInferenceProviderConfig):
29+
"""Configuration for the DeepSeek inference provider."""
30+
31+
base_url: HttpUrl | None = Field(
32+
default=HttpUrl(os.environ.get("DEEPSEEK_BASE_URL", DEFAULT_BASE_URL)),
33+
description="Base URL for the DeepSeek API",
34+
)
35+
36+
@classmethod
37+
def sample_run_config(cls, api_key: str = "${env.DEEPSEEK_API_KEY:=}", **kwargs) -> dict[str, Any]:
38+
return {
39+
"base_url": DEFAULT_BASE_URL,
40+
"api_key": api_key,
41+
}
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
from collections.abc import AsyncIterator
8+
9+
from ogx.providers.utils.inference.openai_mixin import OpenAIMixin
10+
from ogx_api import (
11+
OpenAIChatCompletion,
12+
OpenAIChatCompletionChunk,
13+
OpenAIChatCompletionRequestWithExtraBody,
14+
OpenAICompletion,
15+
OpenAICompletionRequestWithExtraBody,
16+
OpenAIEmbeddingsRequestWithExtraBody,
17+
OpenAIEmbeddingsResponse,
18+
)
19+
20+
from .config import DeepSeekImplConfig
21+
22+
23+
class DeepSeekInferenceAdapter(OpenAIMixin):
24+
"""Inference adapter for the DeepSeek platform.
25+
26+
DeepSeek exposes an OpenAI-compatible chat completions API, so the shared
27+
`OpenAIMixin` handles requests once pointed at DeepSeek's base URL. See
28+
https://api-docs.deepseek.com/.
29+
"""
30+
31+
config: DeepSeekImplConfig
32+
33+
provider_data_api_key_field: str = "deepseek_api_key"
34+
35+
def get_base_url(self) -> str:
36+
return str(self.config.base_url)
37+
38+
async def openai_chat_completion(
39+
self,
40+
params: OpenAIChatCompletionRequestWithExtraBody,
41+
) -> OpenAIChatCompletion | AsyncIterator[OpenAIChatCompletionChunk]:
42+
if params.response_format is not None and params.response_format.type == "json_schema":
43+
raise ValueError(
44+
"DeepSeek does not support response_format type 'json_schema'. Use 'json_object' or 'text' instead."
45+
)
46+
return await super().openai_chat_completion(params)
47+
48+
async def openai_embeddings(
49+
self,
50+
params: OpenAIEmbeddingsRequestWithExtraBody,
51+
) -> OpenAIEmbeddingsResponse:
52+
raise NotImplementedError("DeepSeek does not expose an embeddings endpoint.")
53+
54+
async def openai_completion(
55+
self,
56+
params: OpenAICompletionRequestWithExtraBody,
57+
) -> OpenAICompletion | AsyncIterator[OpenAICompletion]:
58+
"""DeepSeek does not support the legacy /v1/completions endpoint.
59+
60+
DeepSeek's completion API exists only as a beta FIM feature behind a
61+
separate base URL (https://api.deepseek.com/beta), so it is not
62+
reachable through this adapter's standard API surface.
63+
"""
64+
raise NotImplementedError(
65+
"DeepSeek does not support /v1/completions endpoint. Only /v1/chat/completions is supported."
66+
)

tests/integration/inference/test_openai_completion.py

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ def skip_if_model_doesnt_support_openai_completion(client_with_models, model_id)
5656
# https://go.microsoft.com/fwlink/?linkid=2197993.'}}"}
5757
"remote::llama-openai-compat",
5858
"remote::watsonx", # WatsonX only has /v1/chat/completions, no /v1/completions
59+
"remote::deepseek", # DeepSeek does not support /v1/completions
5960
):
6061
pytest.skip(f"Model {model_id} hosted by {provider.provider_type} doesn't support OpenAI completions.")
6162

@@ -107,6 +108,7 @@ def skip_if_doesnt_support_n(client_with_models, model_id):
107108
"remote::cerebras",
108109
"remote::databricks", # Bad request: parameter "n" must be equal to 1 for streaming mode
109110
"remote::watsonx",
111+
"remote::deepseek", # n > 1 is not supported
110112
):
111113
pytest.skip(f"Model {model_id} hosted by {provider.provider_type} doesn't support n param.")
112114

@@ -146,6 +148,14 @@ def skip_if_provider_doesnt_support_tool_calling(client_with_models, model_id):
146148
pytest.skip(f"Model {model_id} hosted by {provider.provider_type} doesn't support tool calling.")
147149

148150

151+
def skip_if_doesnt_support_json_schema(client_with_models, model_id):
152+
provider = provider_from_model(client_with_models, model_id)
153+
if provider.provider_type in (
154+
"remote::deepseek", # DeepSeek doesn't support response_format type 'json_schema'
155+
):
156+
pytest.skip(f"Model {model_id} hosted by {provider.provider_type} doesn't support json_schema response_format.")
157+
158+
149159
@pytest.mark.parametrize(
150160
"test_case",
151161
[
@@ -783,8 +793,9 @@ def test_openai_chat_completion_with_tool_choice_none(openai_client, text_model_
783793
"inference:chat_completion:structured_output",
784794
],
785795
)
786-
def test_openai_chat_completion_structured_output(openai_client, text_model_id, test_case):
787-
# Note: Skip condition may need adjustment for OpenAI client
796+
def test_openai_chat_completion_structured_output(openai_client, client_with_models, text_model_id, test_case):
797+
skip_if_doesnt_support_json_schema(client_with_models, text_model_id)
798+
788799
class AnswerFormat(BaseModel):
789800
first_name: str
790801
last_name: str
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Copyright (c) The OGX Contributors.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the terms described in the LICENSE file in
5+
# the root directory of this source tree.
6+
7+
import os
8+
from unittest.mock import patch
9+
10+
import pytest
11+
12+
from ogx.core.stack import replace_env_vars
13+
from ogx.providers.remote.inference.deepseek.config import DeepSeekImplConfig
14+
from ogx.providers.remote.inference.deepseek.deepseek import DeepSeekInferenceAdapter
15+
from ogx_api import OpenAICompletionRequestWithExtraBody
16+
17+
18+
class TestDeepSeekConfig:
19+
"""Tests for the DeepSeek inference provider config and adapter wiring."""
20+
21+
def test_default_base_url(self):
22+
config = DeepSeekImplConfig(api_key="test-key")
23+
adapter = DeepSeekInferenceAdapter(config=config)
24+
adapter.provider_data_api_key_field = None
25+
26+
assert adapter.get_base_url() == "https://api.deepseek.com/v1"
27+
28+
def test_custom_base_url_from_config(self):
29+
custom_url = "https://custom.deepseek.com/v1"
30+
config = DeepSeekImplConfig(api_key="test-key", base_url=custom_url)
31+
adapter = DeepSeekInferenceAdapter(config=config)
32+
adapter.provider_data_api_key_field = None
33+
34+
assert adapter.get_base_url() == custom_url
35+
36+
@patch.dict(os.environ, {"DEEPSEEK_BASE_URL": "https://env.deepseek.com/v1"})
37+
def test_base_url_from_environment_variable(self):
38+
config_data = DeepSeekImplConfig.sample_run_config(api_key="test-key")
39+
processed_config = replace_env_vars(config_data)
40+
config = DeepSeekImplConfig.model_validate(processed_config)
41+
42+
assert str(config.base_url) == "https://api.deepseek.com/v1"
43+
44+
def test_sample_run_config_uses_env_placeholder(self):
45+
cfg = DeepSeekImplConfig.sample_run_config()
46+
assert cfg["base_url"] == "https://api.deepseek.com/v1"
47+
assert cfg["api_key"] == "${env.DEEPSEEK_API_KEY:=}"
48+
49+
def test_provider_data_api_key_field(self):
50+
config = DeepSeekImplConfig(api_key="test-key")
51+
adapter = DeepSeekInferenceAdapter(config=config)
52+
assert adapter.provider_data_api_key_field == "deepseek_api_key"
53+
54+
async def test_embeddings_not_supported(self):
55+
config = DeepSeekImplConfig(api_key="test-key")
56+
adapter = DeepSeekInferenceAdapter(config=config)
57+
with pytest.raises(NotImplementedError, match="does not expose an embeddings endpoint"):
58+
await adapter.openai_embeddings(None) # type: ignore[arg-type]
59+
60+
async def test_legacy_completions_endpoint_not_supported(self):
61+
"""DeepSeek does not support the legacy /v1/completions endpoint."""
62+
config = DeepSeekImplConfig(api_key="test-key")
63+
adapter = DeepSeekInferenceAdapter(config=config)
64+
65+
params = OpenAICompletionRequestWithExtraBody(model="deepseek-chat", prompt="Hello")
66+
67+
with pytest.raises(NotImplementedError, match="does not support /v1/completions endpoint"):
68+
await adapter.openai_completion(params)

0 commit comments

Comments
 (0)