Skip to content

Commit 52a024d

Browse files
abhishek-mittalclaudeAbhishek Mittalchopratejas
authored
fix(proxy): strip [1m] model suffix before upstream forwarding (headroomlabs-ai#2027)
## Description Scopes the `[1m]` context-window tier suffix sanitizer to Anthropic `/v1/messages` requests only (addresses PR headroomlabs-ai#2027 review feedback). The original patch applied the rewrite to every buffered compressible endpoint, which would have silently mutated OpenAI Chat Completions and OpenAI Responses request model IDs. The `[1m]` marker is an Anthropic/Claude Code compatibility signal emitted by the Headroom CLI; the existing Python parity behavior (`sanitize_anthropic_model_id()`) is Anthropic-specific and must not leak onto OpenAI shapes. Refactors the helper into `compression::sanitize_anthropic_model_id_in_body`, drops the dead `sanitize_model_id` helper in `sse/anthropic.rs`, and adds 8 unit tests + 5 wiremock-backed integration tests that pin the scope. All 420 `headroom-proxy` tests pass; `cargo fmt` and `cargo clippy -D warnings` clean. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Move `sanitize_request_model_id` out of `proxy.rs` and into `compression::sanitize_anthropic_model_id_in_body` (Anthropic-specific name; private `trim_anthropic_model_id_suffix` helper for unit-testable pure behavior). - Gate the call site on `CompressibleEndpoint::AnthropicMessages` **after** classification. The OpenAI Chat Completions and OpenAI Responses arms get an explicit no-op match so the sanitizer cannot re-apply to those paths. - Drop the dead `sanitize_model_id` helper in `sse/anthropic.rs` (it was `#[allow(dead_code)]` with no callers). - 8 new unit tests in `compression/mod.rs`: trailing `[1m]` stripped, Claude-style suffix stripped, no-suffix passthrough (byte-equal), non-string model, missing `model` field, non-JSON body, `[1m]` mid-string, and the pure trim helper. - 5 new integration tests in `tests/integration_anthropic_model_sanitize.rs` that boot a real Rust proxy in front of a wiremock upstream. ## Testing - [x] Unit tests pass (`cargo test -p headroom-proxy` → 420 passed, 35 suites) - [x] Linting passes (`cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` clean) - [x] Type checking passes (`cargo check -p headroom-proxy --tests --all-features` clean) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-proxy --test integration_anthropic_model_sanitize Compiling headroom-proxy v0.x.x Finished `test` profile [unoptimized + debuginfo] target(s) Running tests/integration_anthropic_model_sanitize.rs test anthropic_messages_strips_1m_suffix_glm ... ok test anthropic_messages_strips_1m_suffix_claude ... ok test anthropic_messages_passthrough_when_no_suffix ... ok test openai_chat_completions_passthrough_with_1m_model ... ok test openai_responses_passthrough_with_1m_model ... ok test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ``` ```text $ cargo test -p headroom-proxy test result: ok. 420 passed; 0 failed; 0 ignored; 0 measured; 235 filtered out finished in 10.93s ``` ```text $ cargo clippy -p headroom-proxy --tests --all-features -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) ``` ## Real Behavior Proof - **Environment:** macOS 14.x; `rustc` pinned via `rust-toolchain.toml`; `cargo` 1.x. No network access required (wiremock upstream). - **Exact command / steps:** 1. `cargo test -p headroom-proxy --test integration_anthropic_model_sanitize` — confirms `/v1/messages` strips `glm-5.2[1m]` and `claude-3-7-sonnet[1m]`; confirms `/v1/chat/completions` and `/v1/responses` leave the body byte-equal (SHA-256 asserted). 2. `cargo test -p headroom-proxy` — full suite green (420 passed). 3. `cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` — clean. 4. `cargo fmt -p headroom-proxy --check` — clean. 5. Source inspection of `crates/headroom-proxy/src/proxy.rs` after the change: the call site is now in a `match endpoint` arm that explicitly returns `buffered` for the OpenAI variants, so the sanitizer cannot re-apply to those paths. - **Observed result:** all 5 new integration tests pass, all 420 crate tests pass, clippy and fmt clean. The OpenAI tests assert SHA-256 byte equality on a body whose `model` field ends in `[1m]`; if the sanitizer were to re-leak onto OpenAI shapes these would fail loudly with a length delta. - **Not tested:** a live Anthropic API call (would require real credentials and is not required to prove the byte-level scope fix). The Python proxy's `sanitize_anthropic_model_id()` is the documented parity reference (Python PR headroomlabs-ai#1840, issue headroomlabs-ai#1812). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (N/A — no user-facing docs change; the Python proxy's `sanitize_anthropic_model_id` is the parity reference cited in code comments) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (project uses git log + PR titles; this PR's title follows the conventional commit shape) ## Screenshots (if applicable) N/A — backend behavior, no UI change. ## Additional Notes - The OpenAI integration tests rely on a JWT-style `Authorization: Bearer` header to classify the request as `AuthMode::OAuth` and short-circuit the PR-E4 `prompt_cache_key` injector. This is the same control variable the existing `integration_chat_completions.rs` tests use to isolate dispatcher byte-fidelity from the E4 hook. Comments in each test explain the relationship. - The dead helper in `sse/anthropic.rs` is removed, so the diff is net negative on LoC for the SSE module. - The Python parity reference is `sanitize_anthropic_model_id()` (Python PR headroomlabs-ai#1840, issue headroomlabs-ai#1812); the function name and the call-site scope are the explicit parity contract. - Branch was rebased onto `upstream/main` (91 commits behind) before force-push to the fork; conflict-free rebase. The original PR commit and the fix are the only two commits on the PR. --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: Abhishek Mittal <abhishek.mittal@users.noreply.github.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
1 parent 35701ce commit 52a024d

3 files changed

Lines changed: 507 additions & 1 deletion

File tree

crates/headroom-proxy/src/compression/mod.rs

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,67 @@ pub fn classify_compressible_path(path: &str) -> Option<CompressibleEndpoint> {
8585
}
8686
}
8787

88+
/// Strip the `[1m]` context-window tier suffix that the Headroom
89+
/// CLI appends to Anthropic model IDs (e.g. `glm-5.2[1m]`,
90+
/// `claude-3-7-sonnet[1m]`) before forwarding to the upstream
91+
/// Anthropic API. The upstream does not recognize the suffix and
92+
/// rejects requests with 400.
93+
///
94+
/// This mirrors the Python proxy's `sanitize_anthropic_model_id()`
95+
/// (Python PR #1840, fixes issue #1812). The Rust proxy
96+
/// intentionally scopes this to **Anthropic** requests only — the
97+
/// `[1m]` marker is an Anthropic / Claude Code compatibility
98+
/// signal emitted by the CLI, and we must not silently mutate
99+
/// OpenAI-compatible request model IDs. The caller is responsible
100+
/// for gating on `CompressibleEndpoint::AnthropicMessages` before
101+
/// invoking this function; doing so at the call site (not inside
102+
/// this function) keeps the helper cheap and unambiguous.
103+
///
104+
/// Returns the original `body` byte-for-byte on every "no
105+
/// sanitization" path:
106+
/// - body is not valid JSON
107+
/// - body has no top-level `model` field
108+
/// - top-level `model` is not a string
109+
/// - the string does not end in `[1m]`
110+
/// - re-serialization fails (extremely unusual; we have already
111+
/// parsed the body, but `serde_json` could in theory reject a
112+
/// value shape it accepted on parse)
113+
pub fn sanitize_anthropic_model_id_in_body(body: bytes::Bytes) -> bytes::Bytes {
114+
let Ok(mut parsed) = serde_json::from_slice::<serde_json::Value>(&body) else {
115+
return body;
116+
};
117+
118+
let Some(model_value) = parsed.get_mut("model") else {
119+
return body;
120+
};
121+
122+
let serde_json::Value::String(model) = model_value else {
123+
return body;
124+
};
125+
126+
let sanitized = trim_anthropic_model_id_suffix(model);
127+
if sanitized == *model {
128+
// Either no `[1m]` suffix to strip, or the suffix is not at
129+
// the tail (e.g. `claude-3-7-sonnet[1m]-thinking` — not a
130+
// CLI-emitted shape, but a defensive no-op). Byte-equal
131+
// passthrough preserves the cache-safety invariant.
132+
return body;
133+
}
134+
135+
*model = sanitized;
136+
match serde_json::to_vec(&parsed) {
137+
Ok(buf) => bytes::Bytes::from(buf),
138+
Err(_) => body,
139+
}
140+
}
141+
142+
/// Pure helper: strip a trailing `[1m]` from a model ID string.
143+
/// Exposed for unit tests; production callers should use
144+
/// [`sanitize_anthropic_model_id_in_body`].
145+
fn trim_anthropic_model_id_suffix(model: &str) -> String {
146+
model.trim_end_matches("[1m]").to_string()
147+
}
148+
88149
#[cfg(test)]
89150
mod tests {
90151
use super::*;
@@ -124,4 +185,101 @@ mod tests {
124185
assert!(!is_compressible_path("/"));
125186
assert!(!is_compressible_path(""));
126187
}
188+
189+
// ─── [1m] Anthropic model-suffix sanitizer (PR #2027) ────────────
190+
//
191+
// PR #2027 review feedback (JerrettDavis): the sanitizer must
192+
// strip the `[1m]` CLI suffix from Anthropic `/v1/messages`
193+
// request bodies without mutating OpenAI-shaped bodies. These
194+
// unit tests pin the pure helper; integration coverage for
195+
// /v1/messages vs /v1/chat/completions vs /v1/responses lives
196+
// in `tests/integration_anthropic_model_sanitize.rs`.
197+
198+
#[test]
199+
fn sanitizer_strips_trailing_1m_suffix() {
200+
let body = br#"{"model":"glm-5.2[1m]","max_tokens":1024,"messages":[]}"#;
201+
let out = sanitize_anthropic_model_id_in_body(bytes::Bytes::copy_from_slice(body));
202+
let parsed: serde_json::Value = serde_json::from_slice(&out).unwrap();
203+
assert_eq!(parsed["model"], "glm-5.2");
204+
// Other fields round-trip unchanged.
205+
assert_eq!(parsed["max_tokens"], 1024);
206+
assert!(parsed["messages"].is_array());
207+
}
208+
209+
#[test]
210+
fn sanitizer_strips_suffix_from_claude_model() {
211+
let body = br#"{"model":"claude-3-7-sonnet[1m]","max_tokens":1024}"#;
212+
let out = sanitize_anthropic_model_id_in_body(bytes::Bytes::copy_from_slice(body));
213+
let parsed: serde_json::Value = serde_json::from_slice(&out).unwrap();
214+
assert_eq!(parsed["model"], "claude-3-7-sonnet");
215+
}
216+
217+
#[test]
218+
fn sanitizer_passthrough_when_no_suffix() {
219+
let body = br#"{"model":"claude-3-7-sonnet","max_tokens":1024}"#;
220+
let original = bytes::Bytes::copy_from_slice(body);
221+
let out = sanitize_anthropic_model_id_in_body(original.clone());
222+
// Byte-equal — required to keep the cache-safety invariant
223+
// for the (very common) no-suffix case.
224+
assert_eq!(out, original);
225+
}
226+
227+
#[test]
228+
fn sanitizer_passthrough_when_model_not_string() {
229+
// Anthropic `/v1/messages` requires a string model, but a
230+
// malformed body must round-trip unchanged — we never
231+
// mutate non-string `model` values.
232+
let body = br#"{"model":42,"max_tokens":1024}"#;
233+
let original = bytes::Bytes::copy_from_slice(body);
234+
let out = sanitize_anthropic_model_id_in_body(original.clone());
235+
assert_eq!(out, original);
236+
}
237+
238+
#[test]
239+
fn sanitizer_passthrough_when_no_model_field() {
240+
let body = br#"{"max_tokens":1024,"messages":[]}"#;
241+
let original = bytes::Bytes::copy_from_slice(body);
242+
let out = sanitize_anthropic_model_id_in_body(original.clone());
243+
assert_eq!(out, original);
244+
}
245+
246+
#[test]
247+
fn sanitizer_passthrough_for_non_json_body() {
248+
// A non-JSON body (e.g. SSE-rewritten or streaming chunk)
249+
// must round-trip unchanged. The compressible-path gate in
250+
// proxy.rs already filters by Content-Type=application/json,
251+
// so this is a belt-and-braces check.
252+
let body = b"not json at all";
253+
let original = bytes::Bytes::copy_from_slice(body);
254+
let out = sanitize_anthropic_model_id_in_body(original.clone());
255+
assert_eq!(out, original);
256+
}
257+
258+
#[test]
259+
fn sanitizer_passthrough_for_1m_not_at_tail() {
260+
// Defensive: a `[1m]` mid-string is not a CLI suffix and
261+
// must not be mutated. Real CLI behavior only ever appends
262+
// `[1m]` at the tail, but a future wire format should not
263+
// silently corrupt legitimate `model` strings.
264+
let body = br#"{"model":"claude-3-7-sonnet[1m]-thinking","max_tokens":1024}"#;
265+
let original = bytes::Bytes::copy_from_slice(body);
266+
let out = sanitize_anthropic_model_id_in_body(original.clone());
267+
assert_eq!(out, original);
268+
}
269+
270+
#[test]
271+
fn trim_helper_pure() {
272+
// The pure helper is a thin wrapper around
273+
// `str::trim_end_matches`; pin its behavior directly so
274+
// refactors that swap the implementation can't silently
275+
// change the strip semantics.
276+
assert_eq!(trim_anthropic_model_id_suffix("glm-5.2[1m]"), "glm-5.2");
277+
assert_eq!(trim_anthropic_model_id_suffix("claude[1m]"), "claude");
278+
assert_eq!(trim_anthropic_model_id_suffix("claude"), "claude");
279+
// `trim_end_matches` is greedy — back-to-back suffixes are
280+
// all stripped. The CLI never emits this, but pinning the
281+
// behavior makes the helper a pure function with a stable
282+
// contract.
283+
assert_eq!(trim_anthropic_model_id_suffix("claude[1m][1m]"), "claude");
284+
}
127285
}

crates/headroom-proxy/src/proxy.rs

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ use axum::http::{HeaderMap, HeaderName, Request, Response, StatusCode, Uri};
1010
use axum::response::IntoResponse;
1111
use axum::routing::{any, get, post};
1212
use axum::Router;
13+
use futures_util::{StreamExt as _, TryStreamExt};
1314
#[cfg(test)]
1415
use bytes::Bytes;
15-
use futures_util::{StreamExt as _, TryStreamExt};
1616
#[cfg(test)]
1717
use http_body_util::BodyExt;
1818

@@ -632,6 +632,26 @@ pub(crate) async fn forward_http(
632632
let endpoint = compression::classify_compressible_path(uri.path())
633633
.expect("is_compressible_path guarded above");
634634

635+
// PR-2027: strip the `[1m]` context-window tier suffix from
636+
// the request body for Anthropic messages only. The
637+
// Headroom CLI appends `[1m]` to model IDs (e.g.
638+
// `glm-5.2[1m]`, `claude-3-7-sonnet[1m]`) to signal 1M
639+
// context to Claude Code; the upstream Anthropic API does
640+
// not recognize the suffix and rejects the request. The
641+
// suffix is an Anthropic/Claude Code compatibility marker,
642+
// so we must not silently mutate OpenAI-compatible
643+
// request model IDs. The sanitizer is gated on the
644+
// already-classified `endpoint`, which is the same source
645+
// of truth the dispatcher uses below — keeping the gate
646+
// and the dispatch in lockstep.
647+
let buffered = match endpoint {
648+
compression::CompressibleEndpoint::AnthropicMessages => {
649+
compression::sanitize_anthropic_model_id_in_body(buffered)
650+
}
651+
compression::CompressibleEndpoint::OpenAiChatCompletions
652+
| compression::CompressibleEndpoint::OpenAiResponses => buffered,
653+
};
654+
635655
// PR-E5 + PR-E6: cache-stabilization observability hooks.
636656
// Both run READ-ONLY against the buffered body and emit
637657
// structured logs only — passthrough invariant from Phase A

0 commit comments

Comments
 (0)