Context
Consumers that test a provider's credentials (e.g. a "Test Connection" button) need to distinguish why a call failed:
- rejected credentials (401) or insufficient permissions (403) → the key is bad
- unreachable endpoint (DNS/timeout/connection) → network problem
- model-level issue (missing/retired/unsupported model, e.g. a provider retiring a model id) → the credentials actually work; only the requested model is unavailable
- throttling (429/quota/resource exhausted) → also proves the credentials work
Today Esperanto surfaces these as a raw exception whose only signal is the stringified message, so consumers resort to substring-matching ("401" in msg, "not found" in msg and "model" in msg, etc.). That's fragile: provider wording changes, and a differently-phrased retirement/deprecation error slips past the match and gets misreported as a broken connection. We just hit exactly this with Google retiring Gemini model ids (downstream: lfnovo/open-notebook#970, and the workaround lfnovo/open-notebook#1035).
Ask
Expose a typed way to classify provider errors, so consumers don't string-match. Some options (whichever fits Esperanto's design):
- A small exception hierarchy raised by the AIFactory model calls, e.g.
AuthenticationError, PermissionError, ModelNotFoundError / ModelUnavailableError, RateLimitError, ConnectionError, ProviderError (catch-all).
- And/or a helper like
classify_error(exc) -> ErrorKind plus the underlying HTTP status code where available.
The key distinction we care about: "did we authenticate and reach the provider?" — anything the provider returned after auth (missing model, rate limit) should be separable from auth/network failures, ideally without inspecting message text.
Happy to help / send a PR if you point me at where provider exceptions are normalized. Not urgent — we have a string-based workaround downstream — but this would let every consumer drop the fragile matching.
Context
Consumers that test a provider's credentials (e.g. a "Test Connection" button) need to distinguish why a call failed:
Today Esperanto surfaces these as a raw exception whose only signal is the stringified message, so consumers resort to substring-matching (
"401" in msg,"not found" in msg and "model" in msg, etc.). That's fragile: provider wording changes, and a differently-phrased retirement/deprecation error slips past the match and gets misreported as a broken connection. We just hit exactly this with Google retiring Gemini model ids (downstream: lfnovo/open-notebook#970, and the workaround lfnovo/open-notebook#1035).Ask
Expose a typed way to classify provider errors, so consumers don't string-match. Some options (whichever fits Esperanto's design):
AuthenticationError,PermissionError,ModelNotFoundError/ModelUnavailableError,RateLimitError,ConnectionError,ProviderError(catch-all).classify_error(exc) -> ErrorKindplus the underlying HTTP status code where available.The key distinction we care about: "did we authenticate and reach the provider?" — anything the provider returned after auth (missing model, rate limit) should be separable from auth/network failures, ideally without inspecting message text.
Happy to help / send a PR if you point me at where provider exceptions are normalized. Not urgent — we have a string-based workaround downstream — but this would let every consumer drop the fragile matching.