Description
On the Gemini native generateContent path, a successful (200) response that triggers a CCR retrieval continuation is masked as a synthetic 502 when the continuation response carries a present-null usage count.
handle_gemini_generate_content (headroom/proxy/handlers/gemini.py) has three sites that read usageMetadata. The initial-response site and the non-CCR site both guard against Gemini returning a present-null count (a key present with a JSON null value, which .get(key, default) returns as None rather than the default). The CCR-continuation site did not:
resp_json = final_resp_json
usage = resp_json.get("usageMetadata", {})
total_input_tokens = usage.get("promptTokenCount", total_input_tokens)
output_tokens = usage.get("candidatesTokenCount", output_tokens)
cache_read_tokens = usage.get("cachedContentTokenCount", cache_read_tokens)When the continuation turn reports "promptTokenCount": null (for example a safety-blocked continuation), total_input_tokens becomes None. The immediately following uncached_input_tokens = max(0, total_input_tokens - cache_read_tokens) and the total_input_tokens > 0 baseline guard then raise TypeError on None arithmetic. That exception is caught by the method's outer except Exception, which returns a 502 JSONResponse and records a provider failure, so a genuinely successful upstream turn is reported to the client as a 502.
Reproduction
Drive handle_gemini_generate_content with:
- an initial 200 response that carries a CCR tool call and a valid
promptTokenCount, and
- a CCR continuation (
ccr_response_handler.handle_response) whose usageMetadata.promptTokenCount is null.
Observed: TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType', and the handler returns 502 instead of 200.
Suggested fix
Read the continuation usage through the same _usage_int guard the two sibling sites use, keeping the pre-continuation count as the fallback:
total_input_tokens = _usage_int(usage.get("promptTokenCount"), total_input_tokens)
output_tokens = _usage_int(usage.get("candidatesTokenCount"), output_tokens)
cache_read_tokens = _usage_int(usage.get("cachedContentTokenCount"), cache_read_tokens)The unguarded site was introduced in #2253 (native CCR retrieval); the present-null guard on the sibling sites landed separately and did not cover it.
Description
On the Gemini native
generateContentpath, a successful (200) response that triggers a CCR retrieval continuation is masked as a synthetic 502 when the continuation response carries a present-null usage count.handle_gemini_generate_content(headroom/proxy/handlers/gemini.py) has three sites that readusageMetadata. The initial-response site and the non-CCR site both guard against Gemini returning a present-null count (a key present with a JSONnullvalue, which.get(key, default)returns asNonerather than the default). The CCR-continuation site did not:When the continuation turn reports
"promptTokenCount": null(for example a safety-blocked continuation),total_input_tokensbecomesNone. The immediately followinguncached_input_tokens = max(0, total_input_tokens - cache_read_tokens)and thetotal_input_tokens > 0baseline guard then raiseTypeErroronNonearithmetic. That exception is caught by the method's outerexcept Exception, which returns a 502 JSONResponse and records a provider failure, so a genuinely successful upstream turn is reported to the client as a 502.Reproduction
Drive
handle_gemini_generate_contentwith:promptTokenCount, andccr_response_handler.handle_response) whoseusageMetadata.promptTokenCountisnull.Observed:
TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType', and the handler returns 502 instead of 200.Suggested fix
Read the continuation usage through the same
_usage_intguard the two sibling sites use, keeping the pre-continuation count as the fallback:The unguarded site was introduced in #2253 (native CCR retrieval); the present-null guard on the sibling sites landed separately and did not cover it.