Skip to content

Commit 2212aa8

Browse files
authored
test(inference): cover provider model validation failure and availability messages (#9281)
## What The Gemini model-validation paths gained a dedicated test suite in #6975, but the **Anthropic** and **NVIDIA Endpoints** paths emit the same message contracts without direct tests — the exact gap the repository's risky-path heuristic keeps noting for `inference` files. ## Changes Five tests in `src/lib/inference/provider-models.test.ts`, each asserting the **exact emitted strings** from `src/lib/inference/provider-models.ts`: | Behavior | Message contract | |---|---| | NVIDIA catalog-fetch failure | `Could not validate model against {buildEndpointUrl}/models: {reason}` | | Gemini availability miss | `Model '{model}' is not available from Google Gemini. Checked {endpointUrl}.` | | Anthropic availability miss | `Model '{model}' is not available from Anthropic. Checked {endpointUrl}/v1/models.` | | Anthropic catalog-fetch failure | `Could not validate model against {endpointUrl}/v1/models: {reason}` | | Anthropic 404/405 graceful gap | returns `{ ok: true, validated: false }` | The tests mirror the existing suite's style (injected `runCurlProbeImpl`, `toEqual` assertions on the full result object). ## Verification - `npx vitest run src/lib/inference/provider-models.test.ts` — **29/29 pass** (24 existing + 5 new) - `npx oxlint` — 0 warnings, 0 errors - `npx oxfmt` — applied and clean Signed-off-by: Ashish Shrees <flowoker1@gmail.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added coverage for provider model validation across NVIDIA, Gemini, and Anthropic. * Verified handling of catalog-fetch failures and missing models. * Confirmed Anthropic 404 responses remain non-blocking. * Ensured structured HTTP status details and endpoint-specific error messages are preserved. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Ashish Shrees <flowoker1@gmail.com>
1 parent 692d1fa commit 2212aa8

1 file changed

Lines changed: 109 additions & 0 deletions

File tree

src/lib/inference/provider-models.test.ts

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import {
1616
fetchNvidiaEndpointModels,
1717
fetchOpenAiLikeModels,
1818
validateAnthropicModel,
19+
validateGeminiModel,
1920
validateNvidiaEndpointModel,
2021
validateOpenAiLikeModel,
2122
} from "./provider-models";
@@ -704,6 +705,114 @@ describe("provider model helpers", () => {
704705
restoreMkdtemp();
705706
}
706707
});
708+
709+
it("fails NVIDIA endpoint validation with the checked endpoint when the catalog fetch fails", () => {
710+
const result = validateNvidiaEndpointModel("nemotron", "nvapi-x", {
711+
runCurlProbeImpl: () => ({
712+
ok: false,
713+
httpStatus: 503,
714+
curlStatus: 7,
715+
body: "",
716+
stderr: "connection refused",
717+
message: "connection refused",
718+
}),
719+
});
720+
721+
expect(result).toEqual({
722+
ok: false,
723+
httpStatus: 503,
724+
curlStatus: 7,
725+
message: `Could not validate model against ${BUILD_ENDPOINT_URL}/models: connection refused`,
726+
});
727+
});
728+
729+
it("reports Gemini models missing from the native catalog", () => {
730+
const result = validateGeminiModel("gemini-9.9-missing", "AIzaFakeKey123", {
731+
runCurlProbeImpl: () => ({
732+
ok: true,
733+
httpStatus: 200,
734+
curlStatus: 0,
735+
body: JSON.stringify({
736+
models: [
737+
{
738+
name: "models/gemini-2.5-pro",
739+
supportedGenerationMethods: ["generateContent"],
740+
},
741+
],
742+
}),
743+
stderr: "",
744+
message: "",
745+
}),
746+
});
747+
748+
expect(result).toEqual({
749+
ok: false,
750+
httpStatus: 200,
751+
curlStatus: 0,
752+
message: `Model 'gemini-9.9-missing' is not available from Google Gemini. Checked ${GEMINI_NATIVE_MODELS_ENDPOINT_URL}.`,
753+
});
754+
});
755+
756+
it("reports Anthropic models missing from the catalog", () => {
757+
const result = validateAnthropicModel(
758+
"https://api.anthropic.com",
759+
"claude-missing",
760+
"sk-ant-x",
761+
{
762+
runCurlProbeImpl: () => ({
763+
ok: true,
764+
httpStatus: 200,
765+
curlStatus: 0,
766+
body: JSON.stringify({ data: [{ id: "claude-opus" }] }),
767+
stderr: "",
768+
message: "",
769+
}),
770+
},
771+
);
772+
773+
expect(result).toEqual({
774+
ok: false,
775+
httpStatus: 200,
776+
curlStatus: 0,
777+
message:
778+
"Model 'claude-missing' is not available from Anthropic. Checked https://api.anthropic.com/v1/models.",
779+
});
780+
});
781+
782+
it("fails Anthropic validation with the checked endpoint when the catalog fetch fails", () => {
783+
const result = validateAnthropicModel("https://api.anthropic.com", "claude-opus", "sk-ant-x", {
784+
runCurlProbeImpl: () => ({
785+
ok: false,
786+
httpStatus: 500,
787+
curlStatus: 0,
788+
body: "",
789+
stderr: "HTTP 500",
790+
message: "HTTP 500",
791+
}),
792+
});
793+
794+
expect(result).toEqual({
795+
ok: false,
796+
httpStatus: 500,
797+
curlStatus: 0,
798+
message: "Could not validate model against https://api.anthropic.com/v1/models: HTTP 500",
799+
});
800+
});
801+
802+
it("treats Anthropic 404 catalog responses as non-blocking validation gaps", () => {
803+
const result = validateAnthropicModel("https://api.anthropic.com", "claude-opus", "sk-ant-x", {
804+
runCurlProbeImpl: () => ({
805+
ok: false,
806+
httpStatus: 404,
807+
curlStatus: 0,
808+
body: "",
809+
stderr: "HTTP 404",
810+
message: "HTTP 404",
811+
}),
812+
});
813+
814+
expect(result).toEqual({ ok: true, validated: false });
815+
});
707816
});
708817

709818
function stubFsMkdtempToThrow(): () => void {

0 commit comments

Comments
 (0)