Skip to content

Commit 7708662

Browse files
committed
fix(inference): refresh NVIDIA Endpoints featured models
Signed-off-by: Ho Lim <subhoya@gmail.com>
1 parent e4b9111 commit 7708662

9 files changed

Lines changed: 268 additions & 41 deletions

File tree

ci/test-file-size-budget.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
"test/install-preflight.test.ts": 4006,
1111
"test/nemoclaw-start.test.ts": 5043,
1212
"test/onboard-messaging.test.ts": 2062,
13-
"test/onboard-selection.test.ts": 6867,
13+
"test/onboard-selection.test.ts": 6864,
1414
"test/onboard.test.ts": 4774,
1515
"test/policies.test.ts": 2489
1616
}

docs/inference/inference-options.mdx

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -61,10 +61,12 @@ The first six are always available.
6161
Ollama appears when you have installed or started it on the host.
6262
Local vLLM appears when NemoClaw detects a running vLLM server.
6363
The managed install/start vLLM entry appears by default on DGX Spark and DGX Station, and appears on generic Linux NVIDIA GPU hosts after opt-in.
64+
During interactive NVIDIA Endpoints onboarding, NemoClaw fetches NVIDIA's featured-models catalog for the model picker.
65+
If the catalog cannot be fetched or returns no safe model IDs, the wizard uses the curated fallback list shown below.
6466

6567
| Option | Description | Curated models |
6668
|--------|-------------|----------------|
67-
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, GLM-5.1, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro |
69+
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, GLM 5.1, Kimi K2.6, Minimax M2.7 |
6870
| OpenAI | Routes to the OpenAI API. Set `OPENAI_API_KEY`. | `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-pro-2026-03-05` |
6971
| Other OpenAI-compatible endpoint | Routes to any server that implements `/v1/chat/completions`. NemoClaw uses `/v1/chat/completions` at runtime by default; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` for proxies that implement it, such as some llama.cpp builds. The wizard prompts for a base URL and model name. The adapter is validated against OpenRouter (refer to the status table above); behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations such as LocalAI or llama.cpp may vary. When you enable Telegram messaging, onboarding also runs a bounded sandbox-side smoke check through `https://inference.local/v1/chat/completions`. Set `COMPATIBLE_API_KEY`. | You provide the model name. |
7072
| Anthropic | Routes to the Anthropic Messages API. Set `ANTHROPIC_API_KEY`. | `claude-sonnet-4-6`, `claude-haiku-4-5`, `claude-opus-4-6` |
@@ -78,18 +80,16 @@ The managed install/start vLLM entry appears by default on DGX Spark and DGX Sta
7880

7981
Use this table as starter guidance when selecting a curated cloud model during onboarding.
8082
The provider catalog remains authoritative for exact context-window limits, availability, and current pricing.
83+
The table covers the fallback curated choices; the fetched NVIDIA featured-models catalog can include newer featured models.
8184
The relative labels below are qualitative and compare models within the curated onboarding choices, not across every model a provider offers.
8285

8386
| Model | Best-for task type | Relative latency | Tool-use quality | Context-window fit | Relative cost |
8487
|---|---|---|---|---|---|
85-
| `nvidia/nemotron-3-super-120b-a12b` | Default hosted agent work, multi-step planning, and tool-heavy shell workflows | Medium | Strong default for OpenClaw tool loops | Large agent context | Medium |
8688
| `nvidia/nemotron-3-ultra-550b-a55b` | Quality-sensitive reasoning, careful synthesis, and complex reviews | Higher | Strong for complex tool plans | Large agent context | Higher |
87-
| `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | Reasoning-first and multimodal experiments where a compact hosted model is enough | Medium | Good after the smoke probe confirms final-answer content | Large agent context | Medium |
89+
| `nvidia/nemotron-3-super-120b-a12b` | Default hosted agent work, multi-step planning, and tool-heavy shell workflows | Medium | Strong default for OpenClaw tool loops | Large agent context | Medium |
8890
| `z-ai/glm-5.1` | General chat, multilingual text work, and fast iteration | Low-to-medium | Good for straightforward tool loops | Large agent context | Low-to-medium |
89-
| `minimaxai/minimax-m2.7` | Long-form writing, multi-turn assistant work, and broad instruction following | Medium | Good for structured assistant turns | Large agent context | Medium |
9091
| `moonshotai/kimi-k2.6` | Coding tasks and shell-heavy agent trajectories | Medium | Strong with NemoClaw's Kimi tool-call compatibility path | Large-context friendly | Medium |
91-
| `openai/gpt-oss-120b` | Hosted open-weight style experimentation and cost-aware general agents | Medium | Good when provider-side tool calling is enabled | Large agent context | Medium |
92-
| `deepseek-ai/deepseek-v4-pro` | Code, math, and reasoning-heavy problem solving | Medium-to-high | Strong when the endpoint supports tool calls | Large agent context | Medium-to-high |
92+
| `minimaxai/minimax-m2.7` | Long-form writing, multi-turn assistant work, and broad instruction following | Medium | Good for structured assistant turns | Large agent context | Medium |
9393
| `gpt-5.4` | Default OpenAI-backed agent work and general high-quality reasoning | Medium | Strong | Large agent context | Medium-to-high |
9494
| `gpt-5.4-mini` | Latency-sensitive routine automation and repeated helper calls | Low | Good | Medium-to-large context | Low |
9595
| `gpt-5.4-nano` | Very low-latency classification, routing, extraction, and small helper tasks | Very low | Basic to good for simple tool loops | Medium context | Very low |

src/lib/inference/config.test.ts

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { describe, expect, it } from "vitest";
66
// Import source directly so tests cannot pass against a stale build.
77
import {
88
CLOUD_MODEL_OPTIONS,
9+
DEFAULT_CLOUD_MODEL,
910
DEFAULT_HERMES_PROVIDER_MODEL,
1011
DEFAULT_OLLAMA_MODEL,
1112
DEFAULT_ROUTE_CREDENTIAL_ENV,
@@ -25,16 +26,29 @@ import {
2526

2627
describe("inference selection config", () => {
2728
it("exposes the curated cloud model picker options", () => {
28-
expect(CLOUD_MODEL_OPTIONS.map((option: { id: string }) => option.id)).toEqual([
29-
"nvidia/nemotron-3-super-120b-a12b",
30-
"nvidia/nemotron-3-ultra-550b-a55b",
29+
expect(CLOUD_MODEL_OPTIONS).toEqual([
30+
{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron 3 Ultra 550B" },
31+
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
32+
{ id: "z-ai/glm-5.1", label: "GLM 5.1" },
33+
{ id: "moonshotai/kimi-k2.6", label: "Kimi K2.6" },
34+
{ id: "minimaxai/minimax-m2.7", label: "Minimax M2.7" },
35+
]);
36+
expect(CLOUD_MODEL_OPTIONS.map((option: { id: string }) => option.id)).not.toContain(
3137
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
32-
"z-ai/glm-5.1",
33-
"minimaxai/minimax-m2.7",
34-
"moonshotai/kimi-k2.6",
38+
);
39+
expect(CLOUD_MODEL_OPTIONS.map((option: { id: string }) => option.id)).not.toContain(
3540
"openai/gpt-oss-120b",
41+
);
42+
expect(CLOUD_MODEL_OPTIONS.map((option: { id: string }) => option.id)).not.toContain(
3643
"deepseek-ai/deepseek-v4-pro",
37-
]);
44+
);
45+
});
46+
47+
it("keeps the NVIDIA Endpoints default on Nemotron 3 Super", () => {
48+
expect(DEFAULT_CLOUD_MODEL).toBe("nvidia/nemotron-3-super-120b-a12b");
49+
expect(CLOUD_MODEL_OPTIONS.map((option: { id: string }) => option.id)).toContain(
50+
"nvidia/nemotron-3-super-120b-a12b",
51+
);
3852
});
3953

4054
it("aligns Hermes Provider defaults with the Hermes Agent Nous catalog", () => {

src/lib/inference/config.ts

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -53,14 +53,11 @@ export const HERMES_PROVIDER_MODEL_OPTIONS = [
5353
] as const;
5454
export const DEFAULT_HERMES_PROVIDER_MODEL = HERMES_PROVIDER_MODEL_OPTIONS[0];
5555
export const CLOUD_MODEL_OPTIONS = [
56-
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
5756
{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron 3 Ultra 550B" },
58-
{ id: "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", label: "Nemotron 3 Nano Omni 30B" },
59-
{ id: "z-ai/glm-5.1", label: "GLM-5" },
60-
{ id: "minimaxai/minimax-m2.7", label: "MiniMax M2.7" },
57+
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
58+
{ id: "z-ai/glm-5.1", label: "GLM 5.1" },
6159
{ id: "moonshotai/kimi-k2.6", label: "Kimi K2.6" },
62-
{ id: "openai/gpt-oss-120b", label: "GPT-OSS 120B" },
63-
{ id: "deepseek-ai/deepseek-v4-pro", label: "DeepSeek V4 Pro" },
60+
{ id: "minimaxai/minimax-m2.7", label: "Minimax M2.7" },
6461
];
6562
export const DEFAULT_ROUTE_PROFILE = "inference-local";
6663
export const DEFAULT_ROUTE_CREDENTIAL_ENV = "OPENAI_API_KEY";

src/lib/inference/model-prompts.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,14 +33,14 @@ describe("model prompt helpers", () => {
3333
expect(result).toBe("llama");
3434
});
3535

36-
it("returns DeepSeek V4 Pro from the default cloud model menu", async () => {
37-
const promptFn = promptSequence(["8"]);
36+
it("returns Minimax M2.7 from the default cloud model menu", async () => {
37+
const promptFn = promptSequence(["5"]);
3838
const result = await promptCloudModel({
3939
promptFn,
4040
writeLine: vi.fn(),
4141
});
4242

43-
expect(result).toBe("deepseek-ai/deepseek-v4-pro");
43+
expect(result).toBe("minimaxai/minimax-m2.7");
4444
});
4545

4646
it("validates manual cloud model ids against the saved NVIDIA key", async () => {

src/lib/inference/provider-models.test.ts

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,110 @@ import { describe, expect, it } from "vitest";
66
import {
77
BUILD_ENDPOINT_URL,
88
fetchAnthropicModels,
9+
fetchNvidiaFeaturedModels,
910
fetchNvidiaEndpointModels,
1011
fetchOpenAiLikeModels,
12+
getNvidiaFeaturedModelPromptOptions,
13+
getNvidiaFeaturedModelOptions,
14+
NVIDIA_FEATURED_MODELS_URL,
15+
parseNvidiaFeaturedModels,
1116
validateAnthropicModel,
1217
validateNvidiaEndpointModel,
1318
validateOpenAiLikeModel,
1419
} from "./provider-models";
1520

1621
describe("provider model helpers", () => {
22+
it("parses NVIDIA featured models from the build catalog snapshot shape", () => {
23+
expect(
24+
parseNvidiaFeaturedModels(
25+
JSON.stringify({
26+
"featured-models": [
27+
{
28+
model: "nvidia/nemotron-3-ultra-550b-a55b",
29+
"model-name": "Nemotron 3 Ultra 550B",
30+
},
31+
{
32+
model: "nemotron-3-super-120b-a12b",
33+
"model-name": "Nemotron 3 Super 120B",
34+
},
35+
{ model: "z-ai/glm-5.1", "model-name": "GLM 5.1" },
36+
],
37+
}),
38+
),
39+
).toEqual([
40+
{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron 3 Ultra 550B" },
41+
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
42+
{ id: "z-ai/glm-5.1", label: "GLM 5.1" },
43+
]);
44+
});
45+
46+
it("fetches NVIDIA featured models without requiring an API key", () => {
47+
const result = fetchNvidiaFeaturedModels({
48+
runCurlProbeImpl: (argv) => {
49+
expect(argv.at(-1)).toBe(NVIDIA_FEATURED_MODELS_URL);
50+
expect(argv.join(" ")).not.toContain("Authorization: Bearer");
51+
return {
52+
ok: true,
53+
httpStatus: 200,
54+
curlStatus: 0,
55+
body: JSON.stringify({
56+
"featured-models": [{ model: "moonshotai/kimi-k2.6", "model-name": "Kimi K2.6" }],
57+
}),
58+
stderr: "",
59+
message: "",
60+
};
61+
},
62+
});
63+
64+
expect(result).toEqual({
65+
ok: true,
66+
models: [{ id: "moonshotai/kimi-k2.6", label: "Kimi K2.6" }],
67+
});
68+
});
69+
70+
it("falls back to the curated NVIDIA featured model snapshot when the catalog is unavailable", () => {
71+
const models = getNvidiaFeaturedModelOptions({
72+
runCurlProbeImpl: () => ({
73+
ok: false,
74+
httpStatus: 503,
75+
curlStatus: 0,
76+
body: "",
77+
stderr: "",
78+
message: "HTTP 503",
79+
}),
80+
});
81+
82+
expect(models.map((model) => model.id)).toEqual([
83+
"nvidia/nemotron-3-ultra-550b-a55b",
84+
"nvidia/nemotron-3-super-120b-a12b",
85+
"z-ai/glm-5.1",
86+
"moonshotai/kimi-k2.6",
87+
"minimaxai/minimax-m2.7",
88+
]);
89+
});
90+
91+
it("builds NVIDIA featured model prompt options with the configured default model", () => {
92+
const options = getNvidiaFeaturedModelPromptOptions(null, {
93+
runCurlProbeImpl: () => ({
94+
ok: true,
95+
httpStatus: 200,
96+
curlStatus: 0,
97+
body: JSON.stringify({
98+
"featured-models": [
99+
{ model: "nvidia/nemotron-3-ultra-550b-a55b", "model-name": "Nemotron Ultra" },
100+
],
101+
}),
102+
stderr: "",
103+
message: "",
104+
}),
105+
});
106+
107+
expect(options).toEqual({
108+
defaultModelId: "nvidia/nemotron-3-super-120b-a12b",
109+
cloudModelOptions: [{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron Ultra" }],
110+
});
111+
});
112+
17113
it("fetches NVIDIA endpoint model ids", () => {
18114
const result = fetchNvidiaEndpointModels("nvapi-x", {
19115
runCurlProbeImpl: (argv) => {

src/lib/inference/provider-models.ts

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,23 @@
11
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
// SPDX-License-Identifier: Apache-2.0
33

4+
import { CLOUD_MODEL_OPTIONS, DEFAULT_CLOUD_MODEL } from "./config";
45
import type { CurlProbeResult } from "../adapters/http/probe";
56
import { getCurlTimingArgs, runCurlProbe } from "../adapters/http/probe";
67
import type { ModelCatalogFetchResult, ModelValidationResult } from "../onboard/types";
8+
import { isSafeModelId } from "../validation";
79

810
// credentials.ts still uses CommonJS-style exports.
911
const { normalizeCredentialValue } = require("../credentials/store");
1012

1113
export const BUILD_ENDPOINT_URL = "https://integrate.api.nvidia.com/v1";
14+
export const NVIDIA_FEATURED_MODELS_URL =
15+
"https://assets.ngc.nvidia.com/products/api-catalog/featured-models.json";
1216

1317
export interface ProviderModelOptions {
1418
runCurlProbeImpl?: (argv: string[]) => CurlProbeResult;
1519
buildEndpointUrl?: string;
20+
featuredModelsUrl?: string;
1621
/** When "query-param", send the API key as a ?key= URL parameter instead of
1722
* an Authorization: Bearer header. Required for Google Gemini which rejects
1823
* requests carrying both auth methods. See issue #1960. */
@@ -28,6 +33,32 @@ type ModelCatalogResponse = {
2833
data?: Array<ModelCatalogItem | null>;
2934
};
3035

36+
type FeaturedModelCatalogItem = {
37+
model?: string | null;
38+
"model-name"?: string | null;
39+
};
40+
41+
type FeaturedModelCatalogResponse = {
42+
"featured-models"?: Array<FeaturedModelCatalogItem | null>;
43+
};
44+
45+
export type FeaturedModelOption = {
46+
id: string;
47+
label: string;
48+
};
49+
50+
export type FeaturedModelFetchResult =
51+
| {
52+
ok: true;
53+
models: FeaturedModelOption[];
54+
}
55+
| {
56+
ok: false;
57+
message: string;
58+
httpStatus: number;
59+
curlStatus: number;
60+
};
61+
3162
function parseJson<T>(text: string): T {
3263
return JSON.parse(text);
3364
}
@@ -51,6 +82,98 @@ function parseModelIds(body: string, itemKeys: Array<keyof ModelCatalogItem> = [
5182
.filter((value): value is string => Boolean(value));
5283
}
5384

85+
/**
86+
* Normalizes the NVIDIA featured-models catalog IDs into endpoint model IDs.
87+
*/
88+
function normalizeFeaturedModelId(model: string): string {
89+
const trimmed = model.trim();
90+
if (/^nemotron-3-/i.test(trimmed)) {
91+
return `nvidia/${trimmed}`;
92+
}
93+
return trimmed;
94+
}
95+
96+
/**
97+
* Parses NVIDIA's featured-models catalog into safe onboarding menu options.
98+
*/
99+
export function parseNvidiaFeaturedModels(body: string): FeaturedModelOption[] {
100+
const parsed = parseJson<FeaturedModelCatalogResponse>(body);
101+
const featuredModels = parsed["featured-models"];
102+
if (!Array.isArray(featuredModels)) {
103+
throw new Error('Unexpected featured model catalog response: expected "featured-models" array');
104+
}
105+
106+
return featuredModels
107+
.map((item) => {
108+
const id = typeof item?.model === "string" ? normalizeFeaturedModelId(item.model) : "";
109+
const label = typeof item?.["model-name"] === "string" ? item["model-name"].trim() : "";
110+
return id && label && isSafeModelId(id) ? { id, label } : null;
111+
})
112+
.filter((value): value is FeaturedModelOption => value !== null);
113+
}
114+
115+
/**
116+
* Fetches NVIDIA's public featured-models catalog without requiring credentials.
117+
*/
118+
export function fetchNvidiaFeaturedModels(
119+
options: ProviderModelOptions = {},
120+
): FeaturedModelFetchResult {
121+
const runCurlProbeImpl = options.runCurlProbeImpl ?? runCurlProbe;
122+
const sourceUrl = options.featuredModelsUrl ?? NVIDIA_FEATURED_MODELS_URL;
123+
try {
124+
const result = runCurlProbeImpl([
125+
"-sS",
126+
"--connect-timeout",
127+
"5",
128+
"--max-time",
129+
"15",
130+
sourceUrl,
131+
]);
132+
if (!result.ok) {
133+
return {
134+
ok: false,
135+
message: result.message,
136+
httpStatus: result.httpStatus,
137+
curlStatus: result.curlStatus,
138+
};
139+
}
140+
return { ok: true, models: parseNvidiaFeaturedModels(result.body) };
141+
} catch (error) {
142+
return {
143+
ok: false,
144+
httpStatus: 0,
145+
curlStatus: 0,
146+
message: error instanceof Error ? error.message : String(error),
147+
};
148+
}
149+
}
150+
151+
/**
152+
* Returns live featured NVIDIA models, falling back to the curated snapshot.
153+
*/
154+
export function getNvidiaFeaturedModelOptions(
155+
options: ProviderModelOptions = {},
156+
): FeaturedModelOption[] {
157+
const result = fetchNvidiaFeaturedModels(options);
158+
return result.ok && result.models.length > 0 ? result.models : CLOUD_MODEL_OPTIONS;
159+
}
160+
161+
/**
162+
* Builds NVIDIA Endpoints prompt options from the featured-models catalog.
163+
*/
164+
export function getNvidiaFeaturedModelPromptOptions(
165+
defaultModelId?: string | null,
166+
options: ProviderModelOptions = {},
167+
): {
168+
defaultModelId: string;
169+
cloudModelOptions: FeaturedModelOption[];
170+
} {
171+
return {
172+
defaultModelId: defaultModelId || DEFAULT_CLOUD_MODEL,
173+
cloudModelOptions: getNvidiaFeaturedModelOptions(options),
174+
};
175+
}
176+
54177
function toModelCatalogFetchResult(
55178
result: CurlProbeResult,
56179
itemKeys: Array<keyof ModelCatalogItem> = ["id"],

0 commit comments

Comments
 (0)