fix(grok): route CLI traffic to api.x.ai on shared proxies

[p-clements]

Description

headroom wrap grok points the Grok CLI at the local proxy via GROK_MODELS_BASE_URL and, when it starts the proxy, sets openai_api_url=https://api.x.ai. When Claude/Codex already own port 8787, wrap reuses that process without restart (other wrappers attached), so process-wide OPENAI_API_URL stays https://api.openai.com.

Grok CLI cannot stamp x-headroom-base-url (no custom attribution headers). Inference then goes to OpenAI with xAI session tokens and returns 401. Observed on the wire:

• x-xai-token-auth: xai-grok-cli
• user-agent: grok-pager/... grok-shell/...
• outbound POST https://api.openai.com/v1/chat/completions status 401

This PR adds per-request Grok CLI detection and routes chat / models / responses to https://api.x.ai when those signals are present and the OpenAI target is still its default, without restarting the shared proxy or changing Claude/Codex paths. An operator-configured gateway is never bypassed.

Not covered by #2626 (attribution / UA map only when custom base is already api.x.ai).

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Performance improvement
• Code refactoring (no functional changes)

Changes Made

• Add is_grok_cli_request() in providers/grok/runtime.py using narrow wire signals: exact x-xai-token-auth: xai-grok-cli and the grok-pager/ / grok-shell/ UA product tokens, compared per whitespace-split token so wrappers like litellm-grok/1.0 cannot match
• Resolve OpenAI chat and responses final upstream as: x-headroom-base-url then Grok CLI then process OPENAI_API_URL
• Route GET /v1/models and OpenAI-classified passthrough through the same Grok to xAI base
• Add unit tests for detector, resolver priority, and passthrough/models base selection
• Add tests/test_grok_shared_proxy_call_sites.py, which drives handle_openai_chat and handle_openai_responses against a stubbed transport and asserts the outbound URL — reverting either call site to the old custom_only or OPENAI_API_URL formula turns these red
• Gate Grok to xAI routing on the OpenAI target still being its default, via one shared route_grok_to_xai() predicate used by both the chat/responses resolver and the models/passthrough path, so an operator-configured gateway is never bypassed

Testing

• Unit tests pass (pytest)
• Linting passes (ruff check / ruff format)
• New tests added for new functionality when applicable
• Manual testing performed

Test Output

# Grok routing: call sites, detector, resolver priority, passthrough/models
uv run pytest tests/test_grok_shared_proxy_call_sites.py \
  tests/test_provider_grok.py tests/test_provider_proxy_targets.py \
  tests/test_proxy/test_openai_upstream_header.py \
  tests/test_openai_codex_routing.py -q
============================== 48 passed in 0.46s ==============================

uv run ruff check .
All checks passed!

uv run ruff format --check .
1331 files already formatted

uv run mypy headroom --ignore-missing-imports
Success: no issues found in 506 source files

# Full suite (uv, Python 3.14, tests + scripts/tests) — honesty note
==== 15 failed, 9480 passed, 592 skipped in 553.64s ====
# Replaying those 15 as a set gives an identical 11 failed / 4 passed at the
# merge base, at HEAD~1 and at HEAD, so this branch introduces no failure.
# Causes are local-env: missing cargo, missing litellm extra, Windows-only
# asyncio path, AF_UNIX path length. The 4 test_proxy_health.py kompress tests
# pass in isolation at every ref and fail only under the full suite
# (pre-existing ordering pollution; the new test file is not the cause --
# it runs clean immediately before test_proxy_health.py).
# tests/test_memory_eval.py excluded: it imports litellm at collection time.
# Fork CI pytest not yet run (action_required until maintainer approves).

Real Behavior Proof

• Environment: macOS arm64; Grok CLI 0.2.117; PR branch fix/grok-shared-proxy-xai-routing (0.33.1-dev) running as the only proxy on 8787 with openai_api_url=null and --backend anthropic (same shared-proxy shape as Claude/Codex-started 8787). Stock Headroom 0.33.0 was first used to capture the before failure.
• Exact command / steps: 1) Before: with stock 0.33.0 on 8787, run headroom wrap grok (or attach via GROK_MODELS_BASE_URL) and inspect ~/.headroom/logs/proxy.log. 2) After: replace the process with this branch (uv run python -m headroom.cli proxy --host 127.0.0.1 --port 8787 --backend anthropic, no OpenAI target override). 3) GROK_MODELS_BASE_URL=http://127.0.0.1:8787/v1 grok -p "Reply with exactly one word: pong" --always-approve. 4) Confirm health still shows openai_api_url: null and inspect proxy log outbound lines.
• Observed result: Before (stock): Grok chat with x-xai-token-auth: xai-grok-cli forwarded to https://api.openai.com/v1/chat/completions → 401. After (this branch): same Grok CLI on the same port; /v1/models for user-agent: grok-shell/0.2.117 returned 200; chat outbound was POST path=https://api.x.ai/v1/chat/completions with status 200; CLI printed pong and exited 0. Codex models list against the same proxy still returned 200.
• Not tested: multi-day soak with Claude + Codex + Grok all attached for hours; production PyPI release build; full-suite green under fork CI (local full run had 7 env-only failures unrelated to this change); savings attribution still labels provider as openai without custom base (orthogonal, fix(grok): attribute api.x.ai chat outcomes as provider xai, recognize grok-shell UA #2626).

Review Readiness

• I have performed a self-review
• This PR is ready for human review

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title

Screenshots (if applicable)

N/A — evidence is CLI output and proxy logs (redacted auth).

Additional Notes

• Explicitly out of scope: savings attribution as provider=xai without custom base (see fix(grok): attribute api.x.ai chat outcomes as provider xai, recognize grok-shell UA #2626), multi-provider matrix for Kimi/Mistral, hard-coded per-agent ports.
• ChatGPT Codex auth still wins on /v1/responses before this chain runs.
• Priority: x-headroom-base-url > Grok CLI signals (only while OPENAI_API_URL is still the default) > process OpenAI default. The gate matters because an operator who points the proxy at a gateway (LiteLLM, Azure, self-hosted vLLM) chose it for every OpenAI-compatible client: a client User-Agent must not silently bypass it, nor carry that gateway's OPENAI_TARGET_API_HEADERS to a different vendor. The shared-proxy case this PR fixes runs with openai_api_url: null, so it is unaffected.
• Fork PR workflows may show action_required until a maintainer approves first-time external CI runs.

Runtime Rollout Safety

• Rollout-managed feature(s): none; routing is selected per request from narrow Grok CLI wire signals.
• Minimum rollout channel: normal stable release after focused routing and credential-isolation coverage passes.
• Stable/default behavior changed: only Grok-fingerprinted requests on a proxy whose OpenAI target remains the default route to api.x.ai; explicit client base URLs and operator-configured gateways retain priority.
• Kill switch / disable path: use an explicit non-default OpenAI gateway/target, or human-revert this routing change.
• Unsafe override required: none.
• Qualification impact: Grok chat, Responses, models, passthrough, and cross-vendor header-isolation regressions must remain green.
• Rollback path: human-revert the PR; no persisted state or migration is involved.

[github-actions]

PR governance

This PR follows the template and is marked ready for human review.

[p-clements]

One possible follow-up: if Headroom ends up supporting more black-box CLIs that cannot send x-headroom-base-url, this pattern could be generalised into a small client-fingerprint registry. I kept this PR Grok-specific to avoid scope creep and to make the fix easier to review/revert. The important constraint for any future generalisation is that black-box clients should never need to know about or send x-headroom-base-url.

[JerrettDavis]

The per-request routing fix is well targeted, but it currently crosses an operator credential boundary and needs one change before merge.

Both HTTP handlers build the outbound header set and call merge_extra_headers(headers, self.config.openai_extra_headers) before _resolve_openai_upstream() redirects a Grok-fingerprinted request to https://api.x.ai (chat around 2811-2812/4018; Responses around 4634-4635/4907). Those extras are explicitly OpenAI-target/gateway headers and can include Authorization or other secrets. Because the Grok signals are ordinary client-controlled headers/UA tokens, a request that selects xAI can cause operator-owned OpenAI credentials to be sent to another vendor. This also contradicts the PR's stated invariant that target headers must not cross vendors.

Please make header selection destination-aware: do not merge openai_extra_headers when the resolved destination is xAI (or introduce an xAI-specific header source), for chat, Responses, models, and passthrough as applicable. Add call-site regressions that configure a sentinel OpenAI extra header, send the Grok fingerprint, and prove the outbound xAI request does not contain it while ordinary OpenAI/default and operator-gateway routing still do.

The routing priority/detection behavior otherwise checks out. I ran the 48 focused tests listed in the PR plus Ruff successfully on 13610788; the missing case is credential isolation, not routing functionality. The branch merges cleanly with current main; CI has not been authorized for this fork and I am not asking the contributor to trigger it.

[vercel]

@p-clements is attempting to deploy a commit to the Headroom Labs Team on Vercel.

A member of the Team first needs to authorize it.

[p-clements]

Thanks — you're right. I missed that configured OpenAI extras were merged before destination resolution, so a Grok-fingerprinted request could send OPENAI_TARGET_API_HEADERS (including Authorization) to api.x.ai and clobber the client's credential.

On the direct HTTP path, header selection now uses the resolved OpenAI-compatible upstream host: openai_extra_headers are not merged when that host is api.x.ai. Chat and Responses reuse the same resolved base for isolation and direct URL construction; Responses still gives ChatGPT auth priority for final routing. Models/passthrough do not inject these extras, so no changes were needed there. WebSocket does not use the Grok fingerprint resolver and is unchanged.

Added call-site regressions with sentinel extras for both HTTP handlers, covering xAI isolation, ordinary OpenAI, operator gateways, client-selected xAI, and ChatGPT priority. Ready for re-review.

[JerrettDavis]

Re-reviewed current head b234e24 after the credential-isolation changes. Direct chat and Responses paths now resolve the destination before selecting operator extras, withhold OpenAI-target headers for the official xAI hostname (including ports, paths, and FQDN dots), preserve the client xAI credential, and retain extras for ordinary OpenAI, configured gateways, and configured backend transports. Routing and header policy reuse the same resolved value, eliminating the previous cross-vendor leak. The focused matrix also covers ChatGPT-auth priority and custom bases. CI has not been authorized beyond policy checks; per project policy I am not asking the contributor to trigger it. No remaining source blocker.

[p-clements]

Anything else I can offer with on this?

[chopratejas]

Approved, mergeable, and no failing checks — but CI on this branch only recorded a handful of check runs, well short of the ~35 a full run produces on current main. That usually means the run predates the current workflow set or was path-gated, so "no failures" here isn't the same claim as "green".

I've held it out of today's merge batch for that reason alone, not on content. Please rebase on main (ten PRs landed today) and push so a complete run reports, and I'll merge on the strength of that.

[p-clements]

Updated — branch is now current with main at cb1f53e0.

One note on how: I merged main in rather than rebasing. The previous head ef682d09 was JerrettDavis's own merge commit resolving conflicts in tests/test_openai_codex_routing.py, and rebasing would have rewritten that away and required a force-push. The merge was clean, and it gives the same fresh head for CI.

Locally on the merged tree: 124 passed across the focused set (test_grok_shared_proxy_call_sites.py, test_provider_grok.py, test_provider_proxy_targets.py, test_proxy/test_openai_upstream_header.py, test_openai_codex_routing.py, test_openai_codex_ws_lifecycle.py), plus Ruff and mypy clean.

On the check count, though — I think the diagnosis is slightly off, and it's not something a push can fix. The runs aren't missing or path-gated; they're queued and waiting on approval. Every pull_request workflow on this branch has concluded action_required, across all 47 runs it has accumulated. The push I just made queued another seven in the same state:

CI              :: pull_request        :: action_required
Security        :: pull_request        :: action_required
rust            :: pull_request        :: action_required
Wrap E2E        :: pull_request        :: action_required
Wrap Native E2E :: pull_request        :: action_required
Changelog Guard :: pull_request        :: action_required
Merge Conflicts :: pull_request        :: action_required
PR Governance   :: pull_request_target :: success

The only workflow that executes is PR Governance, because it's pull_request_target and runs in base-repo context — which is where the two checks (template, label) come from. Everything else needs a maintainer to hit Approve and run workflows on the fork. That matches what JerrettDavis noted twice, that CI hasn't been authorized for this fork.

So the branch is current and the head is fresh, but a complete run won't report until someone approves the queued workflows. Happy to push again afterwards if anything needs it.

[p-clements]

You're right — that appeared after my last push. main moved again and #3077 landed in headroom/providers/proxy_targets.py, which is the same file this branch touches. Resolved at f3dc4bf2; the PR reads MERGEABLE again.

Both changes intercept the same OpenAI fall-through in select_passthrough_base_url, so the resolution keeps both:

• The signature takes fix(copilot): route VS Code inline completions to Copilot, not OpenAI #3077's path parameter.
• The Copilot branch runs first. It keys on a path only Copilot emits, so it is the narrower claim on that fall-through; the Grok branch keys on wire signals and only while the OpenAI target is still the default.
• route_grok_to_xai and openai_compatible_base_url are unchanged, so the Grok policy stays defined in one place.

The /v1/alpha/search call site still passes no path, so it skips the Copilot branch on path is not None and resolves exactly as before.

Verification on the merged tree: 154 passed across both sides' focused suites, including all 30 tests from #3077; 667 passed across the routing, passthrough, Copilot and Grok sweep; Ruff and mypy clean. I also checked the Grok branch isn't passing vacuously — deleting it fails test_openai_compatible_base_url_routes_grok_to_xai_on_default_target and nothing else.

Worth noting this file is on a busy path — that conflict opened within a day of the previous merge, so the branch will likely need this again on that cadence.

[JerrettDavis]

Re-reviewed the exact current head after the two main merges. The #3077 conflict resolution preserves the narrower Copilot path claim before Grok fingerprint routing, while explicit x-headroom-base-url/operator targets still outrank Grok and destination-aware header selection still prevents OpenAI/gateway extras from crossing to api.x.ai. The latest merge is otherwise source-neutral for this feature. I also added the missing factual Runtime Rollout Safety section to the PR body. No remaining code blocker; external-fork workflows remain authorization-gated rather than failing.