fix(settings): reliable key validation + deepseek/nvidia probes#21
Open
cmullins70 wants to merge 1 commit into
Open
fix(settings): reliable key validation + deepseek/nvidia probes#21cmullins70 wants to merge 1 commit into
cmullins70 wants to merge 1 commit into
Conversation
…robes
The Anthropic probe was hitting POST /v1/messages with a hardcoded model
("claude-haiku-4-5-20251001"), a real (1-token) inference body, and a 5s
timeout. Newly-created keys whose tier didn't include that model returned
404 (mapped to "unreachable"), TLS cold-start could exceed 5s on slow
networks, and the call cost tokens just to validate.
DeepSeek and NVIDIA were configurable in the UI but the validator
hardcoded the probe-set as {anthropic, openai, groq}, so users with valid
DeepSeek/NVIDIA keys saw a permanent "unchecked" badge.
Changes
- All providers now probe GET /v1/models (free, fast, no model-tier
dependency). Anthropic uses x-api-key + anthropic-version; the rest
use Bearer auth.
- Timeout: 10s total, 5s connect (covers TLS handshake on first probe).
- 403 (revoked/restricted key) now maps to invalid_key alongside 401,
rather than unreachable.
- Added DeepSeek probe -> https://api.deepseek.com/v1/models
- Added NVIDIA probe -> https://integrate.api.nvidia.com/v1/models
- Probe-set centralised in a `probeable` set so the two locations stay
in sync.
Tests
- New TestProviderKeyProbe class (8 cases) covering URL/header shape per
provider, 401/403 -> invalid_key, 5xx -> unreachable, and an explicit
assertion that the probe never POSTs (the original bug).
- Full suite still green (109 passed).
Verified manually in the desktop app with live keys for OpenAI,
Anthropic, DeepSeek, and Groq -- all four now go green on first click.
|
@cmullins70 is attempting to deploy a commit to the Vasu-Devs' projects Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The "Check keys" button in Settings was unreliable for Anthropic and silent for DeepSeek/NVIDIA. This PR fixes both.
What was wrong
Anthropic probe (backend/main.py:1200-1214 on
main) was sending a realPOST /v1/messagesinference call with:claude-haiku-4-5-20251001(returns 404 for keys whose tier doesn't include that exact model → mapped tounreachable),unreachable).For a freshly-created key with
$0credit balance, the result was a "broken" badge on a perfectly valid key.DeepSeek and NVIDIA are first-class providers in the Settings UI (src/settings/shared.tsx:40-47), but the validator's probe-set was hardcoded as
{anthropic, openai, groq}. Users with valid keys saw a permanent neutral "unchecked" badge with no signal whether their key worked.What changed
GET /v1/models— free, fast, no model-tier dependency. Anthropic usesx-api-key+anthropic-version: 2023-06-01; OpenAI/Groq/DeepSeek/NVIDIA use Bearer auth. Endpoint shape verified against the official Anthropic SDK.403now maps toinvalid_keyalongside401(revoked/restricted keys are auth failures, not network issues).api.deepseek.com/v1/models) and NVIDIA (integrate.api.nvidia.com/v1/models).probeableset so_probe_provider_keyandvalidate_settingscan't drift.Tests
New
TestProviderKeyProbeclass inbackend/tests/test_api.py(8 cases, mockshttpx.AsyncClient):x-api-keyvsAuthorization: Bearer).401and403→invalid_key.5xx→unreachable.uv run python -m pytest tests/→ 109 passed, no regressions.Verification
Manually verified in the desktop app via
npm run tauri devagainst live keys for OpenAI, Anthropic, DeepSeek, and Groq. All four now go green on first click.Checklist
pytest backend/tests/— 109 passed.