[BUG] Claude Code session fails with 400 prompt_too_long on 0.35.0 when reading a large JSON file (works direct and on 0.33.0) #3026

Description

When Claude Code reads a moderately large JSON file (~220 KB, well within the model's context window) through the Headroom proxy on 0.35.0, the session consistently fails with an upstream Anthropic API error 400 prompt_too_long — even though the identical session completes successfully when connected directly to api.anthropic.com, and also completes (without compression) through Headroom 0.33.0.

Interestingly, /stats shows that compression did fire on the failing requests (lossless_json, ~14–16k tokens removed per file-bearing request), so the payload sent upstream should have been smaller than the baseline that succeeds. This suggests the rewritten request body is malformed or inflated in a way that trips Anthropic's prompt-length validation, rather than a genuine context overflow.

Thank you for looking into this — happy to provide any additional logs or re-run the repro with debug flags if that helps.

To Reproduce

1. Install: pip install "headroom-ai[proxy]==0.35.0" (fresh venv, Python 3.12.3)
2. Create a synthetic JSON dataset (array of dicts, one record per line):

import json, random
random.seed(42)
regions = ["Hokkaido","Tohoku","Kanto","Chubu","Kansai","Chugoku","Shikoku","Kyushu"]
rows = [{
    "order_id": f"ORD-{100000+i}",
    "date": f"2026-{random.randint(1,7):02d}-{random.randint(1,28):02d}",
    "region": random.choice(regions),
    "product_id": f"PRD-{random.randint(1,80):04d}",
    "units": random.randint(1,50),
    "unit_price_jpy": random.choice([980,1480,2980,4980,9800,14800]),
    "channel": random.choice(["web","store","partner"]),
    "customer_segment": random.choice(["smb","enterprise","consumer"]),
} for i in range(1200)]
with open("data2.json","w") as f:
    f.write("[\n" + ",\n".join(json.dumps(r) for r in rows) + "\n]\n")
# ~220 KB, 1202 lines

3. Also create six tiny notes1.txt … notes6.txt files (3 lines each).
4. Start the proxy: HEADROOM_TELEMETRY=0 headroom proxy --port 18790 (default options)
5. Run a headless Claude Code session through it:

ANTHROPIC_BASE_URL=http://127.0.0.1:18790 claude -p \
  "Do these steps strictly in order, one Read tool call per step: \
   1) Read data2.json in full. 2) Read notes1.txt. 3) Read notes2.txt. \
   4) Read notes3.txt. 5) Read notes4.txt. 6) Read notes5.txt. \
   7) Read notes6.txt. 8) Finally, tell me only what the key fact of notes6.txt is." \
  --model haiku --allowedTools "Read" --max-turns 14 --output-format json

Expected Behavior

The session completes all 8 steps (as it does when ANTHROPIC_BASE_URL is unset), ideally with compression applied to the stale JSON tool_result.

Actual Behavior

The session aborts on the request that carries the JSON tool_result:

• Result JSON: "result": "Prompt is too long", "api_error_status": 400, "terminal_reason": "prompt_too_long", num_turns: 2
• Reproduced 2 out of 2 attempts on 0.35.0, with both a 1,900-row (~350 KB) and a 1,200-row (~220 KB) file. The 1,200-row payload is roughly 60k tokens — nowhere near the model's 200k context window.
• Control runs with the identical prompt and files:
  • Direct to api.anthropic.com (no proxy): completes 8 turns normally.
  • Headroom 0.33.0 (same task): completes 8 turns, though with 0 requests compressed (all skipped as prefix_frozen).

/stats after the failing 0.35.0 runs (cumulative, includes two /v1/compress probes):

"compression": {
  "requests_compressed": 5,
  "avg_compression_pct": 8.0,
  "best_compression_pct": 23.2,
  "best_detail": "130,781 → 100,382 tokens",
  "total_tokens_removed": 60441
},
"uncompressed_requests": { "passthrough": 1, "no_compressible_content": 1 }

So compression fired on the file-bearing requests, yet the upstream rejected them as too long.

Error Output

"result": "Prompt is too long"
"api_error_status": 400
"terminal_reason": "prompt_too_long"

(Claude Code surfaces the upstream Anthropic 400 as the final result; the proxy log shows no error at default verbosity.)

Environment

• Headroom version: 0.35.0 (bug present) / 0.33.0 (no failure, but no compression either)
• Python version: 3.12.3
• OS: Ubuntu (Linux 6.8.0)
• LLM Provider: Anthropic (claude-haiku-4-5-20251001), Claude Code CLI 2.1.231, ChatGPT-style subscription OAuth (claude.ai login), default proxy options
• Health at test time: kompress backend reported degraded (rust_core loaded; SmartCrusher active)

Additional Context

• The same 0.35.0 instance compresses the equivalent payload correctly via a direct POST /v1/compress probe (130,781 → 100,382 tokens, lossless_json), so the transform itself seems fine — the issue appears specific to the live /v1/messages rewrite path with Claude Code's cache_control-laden requests.
• Possibly related to interactions with Claude Code's count_tokens calls (observed as passthrough in stats) or prompt-cache breakpoint handling on the rewritten body.
• Happy to re-run with debug logging or a candidate fix.