Skip to content

Commit c829f4d

Browse files
committed
feat(providers): integrate DeepSeek, NVIDIA NIM, LM Studio, llama.cpp
Add four OpenAI-compatible providers via the AI SDK's createOpenAI().chat() endpoint, each with full BaseProvider semantics, OTEL tracing, error formatting, and validateConfiguration support. Providers: - DeepSeek (api.deepseek.com): cloud, requires DEEPSEEK_API_KEY. - NVIDIA NIM (integrate.api.nvidia.com): cloud, requires NVIDIA_NIM_API_KEY, with NIM-specific extras (top_k, min_p, repetition_penalty, reasoning_budget, chat_template) passed via providerOptions.openai.body and a retry-on-400 path that strips unsupported fields. - LM Studio (localhost:1234): local, auto-discovers loaded model from /v1/models. Optional LM_STUDIO_API_KEY for users behind an auth proxy. - llama.cpp (localhost:8080): local, auto-discovers loaded model. /health validation has 3x retry. Optional LLAMACPP_API_KEY for proxied setups. Credential pass-through verified end-to-end: per-call options.credentials > NeuroLink instance credentials > env vars > documented defaults. Pricing changes (src/lib/utils/pricing.ts): - Add 4 provider entries with symbolic local rates ($1/M tokens) so cost attribution reports a non-zero value after the 6-decimal rounding step. - Treat the `_default` map key as a provider-level fallback (filtered from prefix matches, used as last resort) so providers that don't enumerate per-model pricing still get a non-undefined rate. Context-window root-cause fix (src/lib/constants/contextWindows.ts): - Clamp getOutputReserve to 80% of the context window. Previously, callers that passed maxTokens === contextWindow (or larger) got a 0-token input budget and every request failed with "Budget: 0 tokens" before being sent. Affected 12 test suites and any user passing an oversized maxTokens. BaseProvider change (src/lib/core/baseProvider.ts): - Drop `readonly` on `modelName` so providers that auto-discover the model via /v1/models (lm-studio, llamacpp) can update it after construction, ensuring TelemetryHandler/MessageBuilder cache the resolved name and result.model is never empty. Test infrastructure fixes: - Twelve test files (memory, context, evaluation, mcp, mcp-http, ppt, observability, workflow, tts, media-gen, session-memory-bugs, evaluation-scoring) had a shared `PROVIDER_MAX_TOKENS[provider] || 8192` default. For local providers with 8K context this set maxTokens to the full window → 0 input budget. Fallback lowered to 1024 and the four new providers added explicitly to the per-suite map. - continuous-test-suite-providers.ts: rename `Gemini 3 - DisableTools` test to `DisableTools` and make it provider-agnostic (uses TEST_PROVIDER instead of hard-coded vertex+gemini-3-flash-preview). The Observability Spans test now SKIPs gracefully for Pipeline-A providers (AI SDK + Langfuse OTEL), which intentionally do not emit Pipeline-B spans. - continuous-test-suite-context.ts: Context Compaction Vertex Flash/Pro tests now respect TEST_PROVIDER with vertex as fallback. - continuous-test-suite-evaluation.ts: dimension-specific judge framing in scoreAnswerOnDimension so RAGAS Context Precision actually evaluates the context (not the answer). - continuous-test-suite-client.ts: pass timeout: TEST_CONFIG.timeout to createServer (was using the undocumented 30s default). - continuous-test-suite-observability.ts: read TEST_TIMEOUT_MS from env so CLI-subprocess tests can be tuned for slow local providers. Tooling: - New test/continuous-test-suite-new-providers.ts (10 sections × 4 providers). - New test/run-provider-matrix.sh runner (bash 3.2 compatible). - Add test:dynamic, test:proxy, test:bugfixes, test:new-providers scripts. - Add js-yaml + @types/js-yaml devDeps (proxy Config Loading test fixture). Documentation: - docs/provider-integration/ — 14 markdown files covering architecture, per-provider integration notes, testing methodology, the failure investigation trail, and the PR analysis itself. Verification: - `pnpm run check` 0 errors / 0 warnings (3632 files) - `pnpm run lint` 0 errors / 18 warnings (all pre-existing in unrelated files) - `pnpm run build` clean - All 4 providers smoke-tested end-to-end via env, instance, and per-call credential paths.
1 parent 615f976 commit c829f4d

68 files changed

Lines changed: 10614 additions & 189 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -805,3 +805,62 @@ PICOVOICE_ACCESS_KEY=
805805
# Optional: override LLM provider and model used by the voice pipeline
806806
VOICE_LLM_MODEL=gpt-4o-automatic
807807
VOICE_LLM_PROVIDER=azure
808+
809+
# =============================================================================
810+
# DEEPSEEK CONFIGURATION
811+
# =============================================================================
812+
# Get an API key at https://platform.deepseek.com/api_keys
813+
DEEPSEEK_API_KEY=
814+
# Optional: override default model (deepseek-chat | deepseek-reasoner)
815+
DEEPSEEK_MODEL=deepseek-chat
816+
# Optional: override default base URL
817+
# DEEPSEEK_BASE_URL=https://api.deepseek.com
818+
819+
# =============================================================================
820+
# NVIDIA NIM CONFIGURATION
821+
# =============================================================================
822+
# Get an API key at https://build.nvidia.com/settings/api-keys
823+
NVIDIA_NIM_API_KEY=
824+
# Optional: override default model (browse https://build.nvidia.com/models)
825+
NVIDIA_NIM_MODEL=meta/llama-3.3-70b-instruct
826+
# Optional: override default base URL (use for self-hosted NIM)
827+
# NVIDIA_NIM_BASE_URL=https://integrate.api.nvidia.com/v1
828+
# Optional NIM extras (rarely needed, leave commented)
829+
# NVIDIA_NIM_TOP_K=
830+
# NVIDIA_NIM_MIN_P=
831+
# NVIDIA_NIM_REPETITION_PENALTY=
832+
# NVIDIA_NIM_MIN_TOKENS=
833+
# NVIDIA_NIM_CHAT_TEMPLATE=
834+
835+
# =============================================================================
836+
# LM STUDIO CONFIGURATION (local provider)
837+
# =============================================================================
838+
# Install LM Studio: https://lmstudio.ai/
839+
# Load a model in the app and click "Start Server"
840+
LM_STUDIO_BASE_URL=http://localhost:1234/v1
841+
# Optional: explicit model id (blank = auto-discover from /v1/models)
842+
LM_STUDIO_MODEL=
843+
# Optional: API key (not required for stock LM Studio; use only when running
844+
# behind an auth-proxying reverse-proxy)
845+
# LM_STUDIO_API_KEY=
846+
847+
# =============================================================================
848+
# LLAMA.CPP CONFIGURATION (local provider)
849+
# =============================================================================
850+
# Run: ./llama-server -m model.gguf --port 8080 (add --jinja for tool support)
851+
LLAMACPP_BASE_URL=http://localhost:8080/v1
852+
# Optional: explicit model id (blank = use whatever model llama-server has loaded)
853+
LLAMACPP_MODEL=
854+
# Optional: API key (not required for stock llama-server; use only when running
855+
# behind an auth-proxying reverse-proxy)
856+
# LLAMACPP_API_KEY=
857+
858+
# =============================================================================
859+
# TEST-ONLY CREDENTIALS (used by test/continuous-test-suite-credentials.ts and
860+
# test/continuous-test-suite-new-providers.ts to verify per-call overrides
861+
# without depending on the runtime env vars above)
862+
# =============================================================================
863+
# TEST_DEEPSEEK_API_KEY=
864+
# TEST_NVIDIA_NIM_API_KEY=
865+
# TEST_LM_STUDIO_BASE_URL=
866+
# TEST_LLAMACPP_BASE_URL=

README.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ for await (const chunk of result.stream) {
2626

2727
## 🧠 What is NeuroLink?
2828

29-
**NeuroLink is the universal AI integration platform that unifies 13 major AI providers and 100+ models under one consistent API.**
29+
**NeuroLink is the universal AI integration platform that unifies 17 major AI providers and 100+ models under one consistent API.**
3030

3131
Extracted from production systems at Juspay and battle-tested at enterprise scale, NeuroLink provides a production-ready solution for integrating AI into any application. Whether you're building with OpenAI, Anthropic, Google, AWS Bedrock, Azure, or any of our 13 supported providers, NeuroLink gives you a single, consistent interface that works everywhere.
3232

@@ -195,7 +195,7 @@ npx @juspay/neurolink --help
195195

196196
### Configuration
197197

198-
NeuroLink works with 13+ AI providers. You'll need at least one API key to get started:
198+
NeuroLink works with 17+ AI providers. You'll need at least one API key to get started:
199199

200200
**Option 1: Interactive Setup (Recommended)**
201201

@@ -382,7 +382,7 @@ const result = await neurolink.generate({
382382
### Next Steps
383383

384384
- **[Complete Documentation](https://docs.neurolink.ink)** - Comprehensive guides and API reference
385-
- **[Provider Setup Guide](docs/getting-started/provider-setup.md)** - Configure all 13 providers
385+
- **[Provider Setup Guide](docs/getting-started/provider-setup.md)** - Configure all 17 providers
386386
- **[SDK API Reference](docs/sdk/api-reference.md)** - Full TypeScript API documentation
387387
- **[CLI Command Reference](docs/cli/commands.md)** - Complete CLI documentation
388388
- **[Example Projects](docs/examples/index.md)** - Real-world integration examples
@@ -414,7 +414,7 @@ NeuroLink is a comprehensive AI development platform. Every feature below is pro
414414

415415
### 🤖 AI Provider Integration
416416

417-
**13 providers unified under one API** - Switch providers with a single parameter change.
417+
**17 providers unified under one API** - Switch providers with a single parameter change.
418418

419419
| Provider | Models | Free Tier | Tool Support | Status | Documentation |
420420
| --------------------- | -------------------------------------------------- | --------------- | ------------ | ------------- | ----------------------------------------------------------------------------------------------------------------------------- |
@@ -431,6 +431,10 @@ NeuroLink is a comprehensive AI development platform. Every feature below is pro
431431
| **Ollama** | Local models (Llama, Mistral) | ✅ Free (Local) | ⚠️ Partial | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#ollama) |
432432
| **OpenAI Compatible** | Any OpenAI-compatible endpoint | Varies | ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#openai-compatible) |
433433
| **OpenRouter** | 200+ Models via OpenRouter | Varies | ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/providers/openrouter.md) |
434+
| **DeepSeek** | deepseek-chat (V3), deepseek-reasoner (R1) || ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#deepseek) |
435+
| **NVIDIA NIM** | Llama 3.3 70B, 400+ catalog models || ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#nvidia-nim) |
436+
| **LM Studio** | Any model loaded in LM Studio (local) | ✅ Free (Local) | ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#lm-studio) |
437+
| **llama.cpp** | Any GGUF model served by llama-server (local) | ✅ Free (Local) | ✅ Full | ✅ Production | [Setup Guide](docs/getting-started/provider-setup.md#llamacpp) |
434438

435439
**[📖 Provider Comparison Guide](docs/reference/provider-comparison.md)** - Detailed feature matrix and selection criteria
436440
**[🔬 Provider Feature Compatibility](docs/reference/provider-feature-compatibility.md)** - Test-based compatibility reference for all 19 features across 13 providers
@@ -589,7 +593,7 @@ const result = await neurolink.generate({
589593
- **ProcessorRegistry** - Priority-based processor selection with fallback
590594
- **OWASP Security** - HTML/SVG sanitization prevents XSS attacks
591595
- **Auto-detection** - FileDetector identifies file types by extension and content
592-
- **Provider-agnostic** - All processors work across all 13 AI providers
596+
- **Provider-agnostic** - All processors work across all 17 AI providers
593597

594598
**[📖 File Processors Guide](docs/features/file-processors.md)** - Complete reference for all file types
595599

@@ -731,7 +735,7 @@ Run AI-powered workflows directly in GitHub Actions with 13 provider support and
731735
732736
| Feature | Description |
733737
| ---------------------- | ----------------------------------------------------------------------------------------- |
734-
| **Multi-Provider** | 13 providers with unified interface |
738+
| **Multi-Provider** | 17 providers with unified interface |
735739
| **PR/Issue Comments** | Auto-post AI responses with intelligent updates |
736740
| **Multimodal Support** | Attach images, PDFs, CSVs, Excel, Word, JSON, YAML, XML, HTML, SVG, code files to prompts |
737741
| **Cost Tracking** | Built-in analytics and quality evaluation |
@@ -924,7 +928,7 @@ Full command and API breakdown lives in [`docs/cli/commands.md`](docs/cli/comman
924928
925929
| Capability | Highlights |
926930
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
927-
| **Provider unification** | 13+ providers with automatic fallback, cost-aware routing, provider orchestration (Q3). |
931+
| **Provider unification** | 17+ providers with automatic fallback, cost-aware routing, provider orchestration (Q3). |
928932
| **Multimodal pipeline** | Stream images + CSV data + PDF documents across providers with local/remote assets. Auto-detection for mixed file types. |
929933
| **Quality & governance** | Auto-evaluation engine (Q3), guardrails middleware (Q4), HITL workflows (Q4), audit logging. |
930934
| **Memory & context** | Conversation memory, Redis history export (Q4), context summarization (Q4). |

docs-site/scripts/sync-docs.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1035,6 +1035,14 @@ const LINK_MAPPINGS: Record<string, string> = {
10351035
"providers/openrouter": "/getting-started/providers/openrouter",
10361036
"providers/anthropic": "/getting-started/providers/anthropic",
10371037
anthropic: "/getting-started/providers/anthropic",
1038+
// Bare provider names so sibling-relative `./<name>.md` links inside
1039+
// docs/getting-started/providers/ resolve to the canonical Docusaurus path
1040+
// instead of falling through to the generic `/docs/<name>` (which 404s).
1041+
ollama: "/getting-started/providers/ollama",
1042+
deepseek: "/getting-started/providers/deepseek",
1043+
"nvidia-nim": "/getting-started/providers/nvidia-nim",
1044+
"lm-studio": "/getting-started/providers/lm-studio",
1045+
llamacpp: "/getting-started/providers/llamacpp",
10381046

10391047
// MCP additional
10401048
"mcp/server-catalog": "/guides/mcp/server-catalog",

docs/getting-started/environment-variables.md

Lines changed: 147 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -861,12 +861,138 @@ npx @juspay/neurolink sagemaker setup
861861

862862
---
863863

864+
### 12. DeepSeek
865+
866+
#### Required Variables
867+
868+
```bash
869+
DEEPSEEK_API_KEY="sk-your-deepseek-api-key"
870+
```
871+
872+
#### Optional Variables
873+
874+
```bash
875+
DEEPSEEK_MODEL="deepseek-chat" # Default: deepseek-chat (use deepseek-reasoner for R1)
876+
DEEPSEEK_BASE_URL="https://api.deepseek.com" # Default: DeepSeek API
877+
```
878+
879+
#### How to Get DeepSeek API Key
880+
881+
1. Visit [DeepSeek Platform](https://platform.deepseek.com/api_keys)
882+
2. Sign up or log in to your account
883+
3. Navigate to **API Keys** section
884+
4. Click **Create API Key**
885+
5. Copy the key
886+
887+
#### Supported Models
888+
889+
- `deepseek-chat` (default) - DeepSeek V3, high-quality general chat
890+
- `deepseek-reasoner` - DeepSeek R1, extended chain-of-thought reasoning
891+
892+
---
893+
894+
### 13. NVIDIA NIM
895+
896+
#### Required Variables
897+
898+
```bash
899+
NVIDIA_NIM_API_KEY="nvapi-your-nvidia-api-key"
900+
```
901+
902+
#### Optional Variables
903+
904+
```bash
905+
NVIDIA_NIM_MODEL="meta/llama-3.3-70b-instruct" # Default model
906+
NVIDIA_NIM_BASE_URL="https://integrate.api.nvidia.com/v1" # Default: NVIDIA cloud API (override for self-hosted NIM)
907+
```
908+
909+
#### NIM-Specific Extras (rarely needed)
910+
911+
```bash
912+
# Sampling extras passed as request body extensions
913+
NVIDIA_NIM_TOP_K= # Integer, -1 = disabled (default)
914+
NVIDIA_NIM_MIN_P= # Float, 0 = disabled (default)
915+
NVIDIA_NIM_REPETITION_PENALTY= # Float, 1.0 = disabled (default)
916+
NVIDIA_NIM_MIN_TOKENS= # Integer, 0 = disabled (default)
917+
NVIDIA_NIM_CHAT_TEMPLATE= # Override model chat template string (advanced)
918+
```
919+
920+
#### How to Get NVIDIA NIM API Key
921+
922+
1. Visit [NVIDIA Build](https://build.nvidia.com/)
923+
2. Sign in with your NVIDIA developer account
924+
3. Open **Settings → API Keys**
925+
4. Generate a new API key (Bearer token)
926+
927+
#### Supported Models
928+
929+
- `meta/llama-3.3-70b-instruct` (default) - Llama 3.3 70B Instruct
930+
- Any model listed at [build.nvidia.com/models](https://build.nvidia.com/models)
931+
932+
---
933+
934+
### 14. LM Studio (Local)
935+
936+
LM Studio is a local provider — no API key is required for standard installations.
937+
938+
#### Optional Variables
939+
940+
```bash
941+
LM_STUDIO_BASE_URL="http://localhost:1234/v1" # Default: local LM Studio server
942+
LM_STUDIO_MODEL="" # Blank = auto-discover from /v1/models
943+
# LM_STUDIO_API_KEY= # Only set when running behind an auth-proxying reverse-proxy
944+
```
945+
946+
#### How to Set Up LM Studio
947+
948+
1. Install LM Studio from [lmstudio.ai](https://lmstudio.ai/)
949+
2. Open LM Studio and download a model (e.g., Llama 3.2 3B Instruct)
950+
3. Click **Local Server****Start Server**
951+
4. The server starts at `http://localhost:1234/v1` by default
952+
5. NeuroLink auto-discovers the loaded model; no `LM_STUDIO_MODEL` needed
953+
954+
**Note:** `LM_STUDIO_API_KEY` is only needed if you run LM Studio behind an authenticating reverse-proxy. Vanilla local installs do not require an API key.
955+
956+
---
957+
958+
### 15. llama.cpp (Local)
959+
960+
llama.cpp (llama-server) is a local provider — no API key is required for standard installations.
961+
962+
#### Optional Variables
963+
964+
```bash
965+
LLAMACPP_BASE_URL="http://localhost:8080/v1" # Default: local llama-server
966+
LLAMACPP_MODEL="" # Blank = use whatever model llama-server has loaded
967+
# LLAMACPP_API_KEY= # Only set when running behind an auth-proxying reverse-proxy
968+
```
969+
970+
#### How to Set Up llama.cpp
971+
972+
1. Build llama.cpp from source: [github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp#build)
973+
2. Download a GGUF model file
974+
3. Start llama-server:
975+
976+
```bash
977+
# Basic usage
978+
./llama-server -m model.gguf --port 8080
979+
980+
# With tool/function-call support (required for MCP tools)
981+
./llama-server -m model.gguf --port 8080 --jinja
982+
```
983+
984+
4. NeuroLink auto-discovers the loaded model; no `LLAMACPP_MODEL` needed
985+
986+
**Note:** `LLAMACPP_API_KEY` is only needed if you run llama-server behind an authenticating reverse-proxy. Vanilla local installs do not require an API key.
987+
988+
---
989+
864990
## 🔧 Configuration Examples
865991

866992
### Complete .env File Example
867993

868994
```bash
869-
# NeuroLink Environment Configuration - All 11 Providers
995+
# NeuroLink Environment Configuration - All 15 Providers
870996

871997
# OpenAI Configuration
872998
OPENAI_API_KEY="sk-proj-your-openai-key"
@@ -926,6 +1052,26 @@ MISTRAL_MODEL="mistral-small"
9261052
LITELLM_BASE_URL="http://localhost:4000"
9271053
LITELLM_API_KEY="sk-anything"
9281054
LITELLM_MODEL="openai/gpt-4o-mini"
1055+
1056+
# DeepSeek Configuration
1057+
DEEPSEEK_API_KEY="sk-your-deepseek-key"
1058+
DEEPSEEK_MODEL="deepseek-chat"
1059+
# DEEPSEEK_BASE_URL=https://api.deepseek.com
1060+
1061+
# NVIDIA NIM Configuration
1062+
NVIDIA_NIM_API_KEY="nvapi-your-nvidia-key"
1063+
NVIDIA_NIM_MODEL="meta/llama-3.3-70b-instruct"
1064+
# NVIDIA_NIM_BASE_URL=https://integrate.api.nvidia.com/v1
1065+
1066+
# LM Studio Configuration (local — no API key required)
1067+
LM_STUDIO_BASE_URL="http://localhost:1234/v1"
1068+
# LM_STUDIO_MODEL= # blank = auto-discover
1069+
# LM_STUDIO_API_KEY= # only for reverse-proxy setups
1070+
1071+
# llama.cpp Configuration (local — no API key required)
1072+
LLAMACPP_BASE_URL="http://localhost:8080/v1"
1073+
# LLAMACPP_MODEL= # blank = auto-discover
1074+
# LLAMACPP_API_KEY= # only for reverse-proxy setups
9291075
```
9301076

9311077
### Docker/Container Configuration

0 commit comments

Comments
 (0)