JWT auth error has no flow to get a new token

[Chris Nolan.ca (ChrisNolan)]

In follow up to #1103 scanning the code it seems like there isn't any trigger when a token is given as 'bad' (e.g. via the error_description seen in that other post). A no code solution was to disable that carrier connection and re-enable it which seems to allow the token cache to get a new one, but I'm thinking ideally the auth flow should have this ability to get a new token on error built in? What are others thoughts on that improvement?

(Note: for how I got the bad token... I think it was related to switching between test mode and back again... so there might be another issue with the token cache not fully keyed by the mode perhaps or something? don't need to debug that atm).

[Chris Nolan.ca (ChrisNolan)]

Here's a fast AI summary of the issue ... take it with a grain of salt :shudder:

---

JWT Token Error Masking Issue - FedEx Integration

Problem Summary

When FedEx authentication fails with "Invalid CXS JWT", Karrio's error response masks the underlying JWT issue by returning a generic "service unavailable" error instead. This makes debugging JWT/auth problems difficult.

Current Flow & Root Causes

1. FedEx JWT error isn't captured during shipment creation

   • File: modules/connectors/fedex/karrio/providers/fedex/error.py
   • The error parser only extracts errors from specific response keys (errors, output.alerts, error)
   • OAuth auth errors with error_description field are not parsed/surfaced

2. Auth errors thrown during token refresh aren't retried

   • File: modules/connectors/fedex/karrio/mappers/fedex/proxy.py#L201
   • If authenticate() fails with JWT error, it raises ParsedMessagesError immediately
   • No automatic fallback to re-fetch token even if cached token was bad

3. Token refresh only happens on expiry, not on downstream auth failure

   • File: modules/sdk/karrio/core/utils/caching.py#L182-L210
   • ThreadSafeTokenManager.get_token() checks if token is valid (not expired + buffer)
   • If valid per timestamp, it's reused even if FedEx rejects it downstream
   • No mechanism to detect and retry on "Invalid CXS JWT" from API response

4. Generic service_unavailable error masks JWT issues

   • File: modules/core/karrio/server/core/utils.py#L1094
   • When rate selection fails, system raises generic "service_unavailable"
   • Obscures whether real issue is auth vs actual service unavailability

Current Workarounds

Users can manually force token refresh by:

• Touching/saving the carrier connection (bumps updated_at → new cache version)
• Rotating FedEx API credentials if truly expired/revoked

Suggested Fixes (in priority order)

Short-term (Low effort):

1. Enhance FedEx error parser to extract error_description field → surface auth errors
2. Add error_description handling in modules/connectors/fedex/karrio/providers/fedex/error.py

Medium-term (Medium effort):

1. Implement retry-with-token-refresh on auth failures in proxy layer
2. Detect 401/auth errors in response and clear cache entry to force next authenticate() call
3. Location: modules/connectors/fedex/karrio/mappers/fedex/proxy.py (all lib.request() calls)

Long-term (Higher effort, broader impact):

1. Add generic "retry on auth failure" pattern to base proxy/SDK layer
2. Make ThreadSafeTokenManager listen for downstream 401s and mark token as invalid
3. Files: modules/api/proxy.py, modules/sdk/karrio/core/utils/caching.py

Related Files

• FedEx error handling: modules/connectors/fedex/karrio/providers/fedex/error.py
• FedEx authentication: modules/connectors/fedex/karrio/mappers/fedex/proxy.py#L157-L230
• Token caching logic: modules/sdk/karrio/core/utils/caching.py#L182-L210
• Connection cache versioning: modules/core/karrio/server/providers/models/connection.py#L346, modules/core/karrio/server/providers/models/connection.py#L661
• Service unavailable gate: modules/core/karrio/server/core/utils.py#L1078-L1113

Questions for Team

1. Scope: Is this FedEx-specific or should we fix it carrier-wide?
2. Priority: Is automatic JWT refresh-on-failure worth the extra complexity?
3. Error transparency: Should auth errors be surfaced differently from service errors in API responses?