Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/inference/verify-inference-route.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ $$nemoclaw <name> status

The `Inference` row first checks the sandbox's `inference.local` path.
When that route responds, `status` sends one inference request through the same path.
The row reports `healthy` only when the route returns a structurally valid result for the recorded API family.
When the live provider matches the recorded provider, `status` validates the result against the recorded API family, even when only the model differs.
When the live provider differs, `status` does not carry the recorded API family to the live provider.
The row reports `healthy` only when the route returns a structurally valid result for the selected API family.
An empty body, malformed JSON, provider-error envelope, or wrong response shape reports `unhealthy`, even with a 2xx status.
Status diagnostics do not include the response body.
An HTTP `401` or `403` response reports `unauthorized`.
Expand Down
5 changes: 3 additions & 2 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1876,8 +1876,9 @@ An empty body, malformed JSON, provider-error envelope, or wrong response shape
The probe captures at most 64 KiB and does not include the response body in diagnostics.
The route probe treats any final HTTP status from `200` through `499` as reachable, so a route with an invalidated provider credential answers HTTP `401` while the route is up.
The request uses the live gateway route's provider and model, and falls back to the recorded values when the live route is unreadable.
When the live provider and model match the recorded route, the request also uses the sandbox's recorded API family, including `openai-responses`.
During route drift, NemoClaw does not carry the sandbox's recorded API family to the live provider and model.
When the live provider matches the recorded provider, the request uses the sandbox's recorded API family, even when only the model differs.
This includes `openai-responses`.
When the live provider differs, NemoClaw does not carry the recorded API family to the live provider.
An ordinary run sends one 16-token request through the stored provider credential, with a 30-second timeout, and consumes provider tokens on a hosted route.
When the same `status` run recovers a managed gateway, it retries the route and inference request together up to three total attempts, with a two-second delay between failed attempts.
Each attempt can consume another 16 tokens on a hosted route.
Expand Down
92 changes: 92 additions & 0 deletions src/lib/actions/sandbox/status-snapshot-route-drift.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,95 @@ describe("collectSandboxStatusSnapshot route drift", () => {
expect(snapshot.routeDrift).toMatchObject({ canConnect: false });
});
});

describe("collectSandboxStatusSnapshot inference invocation route (#9302)", () => {
afterEach(() => {
vi.clearAllMocks();
});

/**
* The probe stub rejects a request that does not contain every expected route field.
*/
async function collectInferenceHealth(
entry: Partial<SandboxEntry>,
expectedRoute: Record<string, unknown>,
) {
const sandbox = {
name: "alpha",
agent: "openclaw",
policies: [],
gatewayName: "nemoclaw",
...entry,
} as SandboxEntry;
const snapshot = await collectSandboxStatusSnapshot("alpha", {
deps: {
getSandbox: () => sandbox,
listSandboxes: () => ({ sandboxes: [sandbox], defaultSandbox: "alpha" }),
reconcile: async () => ({ state: "present", output: "Phase: Ready" }),
probeProviderHealthImpl: () => null,
probeSandboxInferenceGatewayHealthImpl: async () => ({
ok: true,
endpoint: "https://inference.local/v1/models",
detail: "reachable",
httpStatus: 200,
}),
probeSandboxInferenceInvocationImpl: (input: Record<string, unknown>) => {
const accepted = Object.entries(expectedRoute).every(
([key, value]) => input[key] === value,
);
return accepted ? { ok: true } : { ok: false, reason: "unexpected invocation route" };
},
},
} as never);
return snapshot.inferenceHealth;
}

const recorded = {
provider: "compatible-endpoint",
model: "recorded/model",
endpointUrl: "https://target.example/v1",
credentialEnv: "TARGET_KEY",
preferredInferenceApi: "openai-responses",
} satisfies Partial<SandboxEntry>;

it("keeps the recorded API family when only the model drifted", async () => {
// The recorded API family describes the provider, which has not changed, so
// dropping it would probe /v1/chat/completions against a responses-only
// endpoint and report a healthy route as unhealthy.
liveGatewayInference("compatible-endpoint", "live/model");

expect(
await collectInferenceHealth(recorded, {
provider: "compatible-endpoint",
model: "live/model",
preferredInferenceApi: "openai-responses",
}),
).toMatchObject({ ok: true, probed: true });
});

it("keeps the recorded API family when the route is aligned", async () => {
liveGatewayInference("compatible-endpoint", "recorded/model");

expect(
await collectInferenceHealth(recorded, {
provider: "compatible-endpoint",
model: "recorded/model",
preferredInferenceApi: "openai-responses",
}),
).toMatchObject({ ok: true, probed: true });
});

it("drops the recorded API family when the provider itself drifted", async () => {
// A provider change must remove the recorded API family because the live
// provider might not implement it.
liveGatewayInference("nvidia-prod", "nvidia/nemotron");

expect(
await collectInferenceHealth(recorded, {
provider: "nvidia-prod",
model: "nvidia/nemotron",
preferredInferenceApi: null,
}),
).toMatchObject({ ok: true, probed: true });
});
});
11 changes: 7 additions & 4 deletions src/lib/actions/sandbox/status-snapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -589,11 +589,14 @@ export async function collectSandboxStatusSnapshot(
? {
provider: live.provider,
model: live.model,
// The live gateway RPC does not expose a stored API override. Do
// not carry an API family across route drift. When the live pair
// is unchanged, the recorded family still describes that route.
// The live gateway RPC does not expose a stored API family. The
// recorded API family describes the recorded provider, so it keeps
// describing the live route while that provider is unchanged,
// including when only the model drifted. Drop it only when the
// provider itself changed, so one provider's API family cannot be
// carried onto another that has no such endpoint (#9302).
preferredInferenceApi:
routeDriftPlan?.kind === "aligned" ? (sb?.preferredInferenceApi ?? null) : null,
live.provider === sb?.provider ? (sb?.preferredInferenceApi ?? null) : null,
}
: {
provider: currentProvider,
Expand Down
Loading