Commit c829f4d
committed
feat(providers): integrate DeepSeek, NVIDIA NIM, LM Studio, llama.cpp
Add four OpenAI-compatible providers via the AI SDK's createOpenAI().chat()
endpoint, each with full BaseProvider semantics, OTEL tracing, error
formatting, and validateConfiguration support.
Providers:
- DeepSeek (api.deepseek.com): cloud, requires DEEPSEEK_API_KEY.
- NVIDIA NIM (integrate.api.nvidia.com): cloud, requires NVIDIA_NIM_API_KEY,
with NIM-specific extras (top_k, min_p, repetition_penalty, reasoning_budget,
chat_template) passed via providerOptions.openai.body and a retry-on-400
path that strips unsupported fields.
- LM Studio (localhost:1234): local, auto-discovers loaded model from
/v1/models. Optional LM_STUDIO_API_KEY for users behind an auth proxy.
- llama.cpp (localhost:8080): local, auto-discovers loaded model. /health
validation has 3x retry. Optional LLAMACPP_API_KEY for proxied setups.
Credential pass-through verified end-to-end: per-call options.credentials >
NeuroLink instance credentials > env vars > documented defaults.
Pricing changes (src/lib/utils/pricing.ts):
- Add 4 provider entries with symbolic local rates ($1/M tokens) so cost
attribution reports a non-zero value after the 6-decimal rounding step.
- Treat the `_default` map key as a provider-level fallback (filtered from
prefix matches, used as last resort) so providers that don't enumerate
per-model pricing still get a non-undefined rate.
Context-window root-cause fix (src/lib/constants/contextWindows.ts):
- Clamp getOutputReserve to 80% of the context window. Previously, callers
that passed maxTokens === contextWindow (or larger) got a 0-token input
budget and every request failed with "Budget: 0 tokens" before being sent.
Affected 12 test suites and any user passing an oversized maxTokens.
BaseProvider change (src/lib/core/baseProvider.ts):
- Drop `readonly` on `modelName` so providers that auto-discover the model
via /v1/models (lm-studio, llamacpp) can update it after construction,
ensuring TelemetryHandler/MessageBuilder cache the resolved name and
result.model is never empty.
Test infrastructure fixes:
- Twelve test files (memory, context, evaluation, mcp, mcp-http, ppt,
observability, workflow, tts, media-gen, session-memory-bugs,
evaluation-scoring) had a shared `PROVIDER_MAX_TOKENS[provider] || 8192`
default. For local providers with 8K context this set maxTokens to the
full window → 0 input budget. Fallback lowered to 1024 and the four new
providers added explicitly to the per-suite map.
- continuous-test-suite-providers.ts: rename `Gemini 3 - DisableTools` test
to `DisableTools` and make it provider-agnostic (uses TEST_PROVIDER instead
of hard-coded vertex+gemini-3-flash-preview). The Observability Spans test
now SKIPs gracefully for Pipeline-A providers (AI SDK + Langfuse OTEL),
which intentionally do not emit Pipeline-B spans.
- continuous-test-suite-context.ts: Context Compaction Vertex Flash/Pro
tests now respect TEST_PROVIDER with vertex as fallback.
- continuous-test-suite-evaluation.ts: dimension-specific judge framing in
scoreAnswerOnDimension so RAGAS Context Precision actually evaluates the
context (not the answer).
- continuous-test-suite-client.ts: pass timeout: TEST_CONFIG.timeout to
createServer (was using the undocumented 30s default).
- continuous-test-suite-observability.ts: read TEST_TIMEOUT_MS from env so
CLI-subprocess tests can be tuned for slow local providers.
Tooling:
- New test/continuous-test-suite-new-providers.ts (10 sections × 4 providers).
- New test/run-provider-matrix.sh runner (bash 3.2 compatible).
- Add test:dynamic, test:proxy, test:bugfixes, test:new-providers scripts.
- Add js-yaml + @types/js-yaml devDeps (proxy Config Loading test fixture).
Documentation:
- docs/provider-integration/ — 14 markdown files covering architecture,
per-provider integration notes, testing methodology, the failure
investigation trail, and the PR analysis itself.
Verification:
- `pnpm run check` 0 errors / 0 warnings (3632 files)
- `pnpm run lint` 0 errors / 18 warnings (all pre-existing in unrelated files)
- `pnpm run build` clean
- All 4 providers smoke-tested end-to-end via env, instance, and per-call
credential paths.1 parent 615f976 commit c829f4d
68 files changed
Lines changed: 10614 additions & 189 deletions
File tree
- docs-site/scripts
- docs
- getting-started
- providers
- provider-integration
- reference
- examples
- memory-bank
- development
- src
- cli
- factories
- utils
- lib
- adapters
- constants
- core
- factories
- providers
- proxy
- types
- utils
- test
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
805 | 805 | | |
806 | 806 | | |
807 | 807 | | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
195 | 195 | | |
196 | 196 | | |
197 | 197 | | |
198 | | - | |
| 198 | + | |
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
| |||
382 | 382 | | |
383 | 383 | | |
384 | 384 | | |
385 | | - | |
| 385 | + | |
386 | 386 | | |
387 | 387 | | |
388 | 388 | | |
| |||
414 | 414 | | |
415 | 415 | | |
416 | 416 | | |
417 | | - | |
| 417 | + | |
418 | 418 | | |
419 | 419 | | |
420 | 420 | | |
| |||
431 | 431 | | |
432 | 432 | | |
433 | 433 | | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
434 | 438 | | |
435 | 439 | | |
436 | 440 | | |
| |||
589 | 593 | | |
590 | 594 | | |
591 | 595 | | |
592 | | - | |
| 596 | + | |
593 | 597 | | |
594 | 598 | | |
595 | 599 | | |
| |||
731 | 735 | | |
732 | 736 | | |
733 | 737 | | |
734 | | - | |
| 738 | + | |
735 | 739 | | |
736 | 740 | | |
737 | 741 | | |
| |||
924 | 928 | | |
925 | 929 | | |
926 | 930 | | |
927 | | - | |
| 931 | + | |
928 | 932 | | |
929 | 933 | | |
930 | 934 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1035 | 1035 | | |
1036 | 1036 | | |
1037 | 1037 | | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
1038 | 1046 | | |
1039 | 1047 | | |
1040 | 1048 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
861 | 861 | | |
862 | 862 | | |
863 | 863 | | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
864 | 990 | | |
865 | 991 | | |
866 | 992 | | |
867 | 993 | | |
868 | 994 | | |
869 | | - | |
| 995 | + | |
870 | 996 | | |
871 | 997 | | |
872 | 998 | | |
| |||
926 | 1052 | | |
927 | 1053 | | |
928 | 1054 | | |
| 1055 | + | |
| 1056 | + | |
| 1057 | + | |
| 1058 | + | |
| 1059 | + | |
| 1060 | + | |
| 1061 | + | |
| 1062 | + | |
| 1063 | + | |
| 1064 | + | |
| 1065 | + | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
| 1070 | + | |
| 1071 | + | |
| 1072 | + | |
| 1073 | + | |
| 1074 | + | |
929 | 1075 | | |
930 | 1076 | | |
931 | 1077 | | |
| |||
0 commit comments