fix(vertex): explain Vertex failures in-band, correct the model/region matrix

[zeroasterisk]

Description

Vertex rejects onboarding requests for a handful of recurring reasons — expired ADC, a partner model never enabled in Model Garden, a model not served in the requested location, a missing optional SDK — and its own error text names none of the fixes. Users see a bare 404 or an opaque 500 through their SDK and cannot tell a proxy bug from a project-config gap.

This makes the proxy explain those failures where it happens, adds a [vertex] extra so the LiteLLM Vertex backend can actually start, and corrects an example, a test matrix, and three documentation claims that could not have worked as written.

Closes #2632

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Performance improvement
• Code refactoring (no functional changes)

Changes Made

Explain Vertex failures in-band (headroom/providers/vertex/diagnostics.py, new)

• On 401/403/404/429, attach an actionable hint three ways: a WARNING in the proxy log, an x-headroom-hint response header, and a note appended to error.message. The last is load-bearing — SDKs raise with the message string and discard headers.
• Successful responses are never modified; annotation is idempotent; statuses we cannot explain get no hint rather than a plausible-sounding wrong one.
• Routes opt in via a one-line @with_vertex_diagnostics decorator reading path params FastAPI already injects (+8 lines in proxy_routes.py, no re-indentation).
• Same treatment for backend setup failures, where --backend vertex surfaced LiteLLM's raw No module named 'vertexai' as an opaque 500.

Make --backend vertex able to start (pyproject.toml, registry.py, cli/proxy.py)

• New [vertex] extra carrying google-cloud-aiplatform>=1.38. Deliberately not folded into [proxy]: measured at ~175 MB installed, and the native passthrough routes never needed it. Follows the existing proxy-prod split.
• Startup preflight keyed on the resolved provider, so every alias normalized at registry.py:229 (vertex, vertex_ai, litellm-vertex, google-vertex, googlevertex) is covered. Skipped when a backend class is injected, so existing fake-based tests are unaffected.
• New BackendUnavailableError so the CLI prints the remedy and exits 2 instead of dumping a traceback through startup internals.

Correct things that could not have worked

	was	is
SDK client	http_options={"api_endpoint": ...}	{"base_url": "http://..."} — api_endpoint is not a field on HttpOptions; pydantic rejects it before any request is sent
Thinking config	ThinkingConfig(thinking_budget_tokens=)	thinking_budget=
Request payload	contents: [{parts: [...]}]	adds role: "user" — the global endpoint enforces it, regional was silently lenient
Auth guidance	gcloud auth print-access-token	application-default — the plain user token gets 401 ACCESS_TOKEN_TYPE_UNSUPPORTED
Test matrix	model × location cross-product	explicit servable pairs

• Test matrix is now four pairs that can actually serve: global/gemini-flash-latest, global/claude-sonnet-4-6, europe-west2/gemini-3.5-flash, us-east5/claude-sonnet-4-6. Gemini 3.x has no US regional endpoint and -latest aliases are global-only, so a pinned us-central1 Gemini case can never pass. No Gemini 2.5 — it retires 2026-10-16.
• test_thinking_levels asserted only status_code < 500 while sending an ignored thinkingBudgetTokens key: it passed on 400s and proved nothing. Now asserts 200 plus scored usage metadata with the correct thinkingBudget spelling.
• Cases the caller's project genuinely cannot serve now skip with an actionable reason instead of reporting a red proxy.

Docs (proxy.mdx, installation.mdx, wiki/vertex.md)

• proxy.mdx claimed publisher=anthropic passthrough "routes through the same LiteLLM-Vertex path as --backend vertex_ai". It does not — without the flag it is a direct forward, which is why it works on a bare [proxy] install where the LiteLLM path cannot even import.
• Documented the trap of combining --backend vertex with native Vertex URLs: LiteLLM takes project/region/model from its own config and ignores the URL.
• installation.mdx and wiki/vertex.md said the SDK is "not included in any extra".
• Cleared 15 pre-existing ruff findings in the two files this PR rewrites (4 remain in plugins/headroom-oauth2, untouched by this work).

Testing

• Unit tests pass (pytest)
• Linting passes (ruff check .)
• Type checking passes (mypy headroom)
• New tests added for new functionality
• Manual testing performed

Test Output

$ mypy headroom
Success: no issues found in 511 source files

$ ruff check .
Found 4 errors.        # all in plugins/headroom-oauth2, pre-existing, untouched by this PR

$ pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q
44 passed, 1 warning in 69.86s          # 8 live integration + 36 offline diagnostics, 0 skipped

$ pytest tests/ -k "anthropic_messages or handler_anthropic or backend or registry or cli_proxy or vertex or proxy_routes"
853 passed, 49 skipped, 9345 deselected, 2 warnings in 28.41s

New tests in tests/test_vertex_diagnostics.py (36, no credentials required): status coverage, per-publisher remedy divergence, idempotency, content-length consistency after body rewrite, non-JSON and body-less/streaming responses, every backend alias preflighted, other providers unaffected, injected backend bypasses the preflight.

Real Behavior Proof

• Environment: macOS 14, Python 3.11.15, live GCP project (alanblount-sandbox) with aiplatform.googleapis.com enabled and Claude enabled in Model Garden; ADC via gcloud auth application-default login
• Exact command / steps: export GCP_PROJECT_ID=alanblount-sandbox && export GCP_ACCESS_TOKEN=$(gcloud auth application-default print-access-token) && uv run --extra proxy --with google-genai python3 examples/vertex_genai_sdk_demo.py && uv run --extra proxy --with pytest-asyncio pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q && headroom proxy --backend vertex (exits 2, then retried with [proxy,vertex] extra installed)
• Observed result: demo printed ✓ Standard response received successfully! > 1, 2, 3, 4, 5 and ✓ Thinking response received successfully!; 44 tests passed (8 live integration, 36 diagnostics), 0 skipped; --backend vertex exited 2 with the remedy message; same request with the extra installed returned 200; invalid region produced [headroom] hint: in the response body and x-headroom-hint header; success responses were byte-clean
• Not tested: Windows; us/eu multi-region endpoints and Claude 4.7+ (project cannot serve them); Provisioned Throughput; Rust proxy path; --backend vertex dropping URL project/model on native Vertex routes (documented as known gap in Additional Notes)

Review Readiness

• I have performed a self-review
• This PR is ready for human review

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

N/A — proxy and CLI behavior; terminal output is quoted above.

Additional Notes

• uv.lock intentionally not included: the new extra is opt-in and does not change default resolution.
• Error-body rewriting is confined to already-failing responses and wrapped so a diagnostic can never turn a clean upstream error into a proxy 500.
• The preflight raises where the provider is resolved (registry.py) rather than at the CLI, so programmatic embedders get the same guarantee; the CLI only formats it.
• Three test files fail to collect on this branch (test_hermes_passthrough_compression, test_observability_metrics, test_observability_tracing). They predate this work and are unrelated, so they were excluded rather than fixed — happy to open a separate issue.
• [vertex] was chosen over adding the dependency to [proxy] after measuring the footprint at ~175 MB. If maintainers prefer it in [proxy], that is a one-line change.

Runtime Rollout Safety

• Rollout-managed feature(s): none; diagnostics annotate already-failing Vertex responses and the backend dependency is opt-in through the vertex extra.
• Minimum rollout channel: normal stable release after Vertex diagnostics, route, and backend-path tests pass.
• Stable/default behavior changed: successful responses and native passthrough remain unchanged; supported Vertex failures gain an actionable header/message, and selecting the LiteLLM Vertex backend without its SDK now fails once at startup.
• Kill switch / disable path: use native Vertex passthrough without the LiteLLM backend, or human-revert the diagnostic/preflight change.
• Unsafe override required: none.
• Qualification impact: offline diagnostics, native integration, and mocked LiteLLM Vertex request-shaping tests must remain green.
• Rollback path: human-revert the PR; no persisted state or migration is involved.

[github-actions]

PR governance

This PR follows the template and is marked ready for human review.

[zeroasterisk]

Integration test against #1044 (headroom wrap agy)

I did a local temp-branch merge of this PR + davdittrich/headroom#1044 to check compatibility before either lands.

Merge result: Two conflicts, both trivial — cli/wrap.py and mcp_registry/install.py each had a region where our branch deleted code that main had already removed, while the agy branch retained and extended it. Resolved by taking the agy version in both cases. No vertex logic is affected.

Shared files that merged cleanly: proxy_routes.py, proxy.mdx, pyproject.toml — non-overlapping changes.

agy unit tests on the merged branch: 97 passed, 5 skipped.
Vertex tests on the merged branch: 42 passed.

One blocker found in #1044 itself: headroom wrap agy fails at startup on the merged branch with:

Error: agy MITM transport failed to start: No module named 'headroom.graph.tokensave_installer'

This module was retired in #2499 (refactor(wrap): retire tokensave; Serena is the code-memory MCP), which merged to main after the agy branch diverged. The agy branch is 122 commits behind main. A rebase to pick up #2499 will be required before #1044 can land — wrap.py in that branch still imports from headroom.graph.tokensave_installer import ensure_tokensave, get_tokensave_path at line ~1011.

Also found one missing re-export: build_tokensave_spec is defined in mcp_registry/install.py but not exposed through mcp_registry/__init__.py, which wrap.py imports from — will cause an ImportError independent of the tokensave module issue.

No action needed on this PR. Flagging here for visibility and tagging #1044.

[JerrettDavis]

Reviewed the current diff with focus on the new Vertex diagnostics, LiteLLM Vertex preflight, route decorators, Anthropic backend error annotation, docs, and the new tests. I did not find a blocking issue in the source changes.\n\nLocal verification: pytest tests/test_vertex_diagnostics.py -q passes (36 passed). I also attempted pytest tests/test_vertex_diagnostics.py tests/test_vertex_claude_compression.py -q; the second module could not collect locally because this checkout does not have the native headroom._core extension built, so I could not use that broader existing test as local evidence.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 85.48387% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
headroom/providers/vertex/diagnostics.py	89.32%	6 Missing and 5 partials ⚠️
headroom/proxy/handlers/anthropic.py	16.66%	5 Missing ⚠️
headroom/cli/proxy.py	33.33%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[JerrettDavis]

Thanks for the Vertex diagnostic work. I need to pull back my earlier approval because fresh CI exposed a branch-local regression.

test (3) is failing on the latest head in two existing LiteLLM Vertex backend-path tests:

FAILED tests/test_openai_streaming_backend.py::TestOpenAIStreamingMock::test_litellm_vertex_streaming_preserves_max_tokens_and_vendor_fields
FAILED tests/test_proxy/test_openai_backend_path.py::test_litellm_vertex_backend_path_preserves_max_tokens_and_vendor_fields
E headroom.providers.registry.BackendUnavailableError: Vertex backend selected but the Vertex SDK is missing...

The new startup preflight is firing before those tests can exercise the mocked/backend-path behavior. The production guard makes sense, but the test surface needs to be updated so CI can still validate the LiteLLM Vertex request-shaping path without requiring google-cloud-aiplatform in the default proxy environment. A small targeted fix would be to inject/mock the backend class or patch the preflight in those existing tests, while keeping the new CLI/startup failure-path coverage for the missing SDK case.

[JerrettDavis]

The previous blocker is fixed at the current head. The new 52686520 commit patches vertex_sdk_available in the two existing LiteLLM Vertex backend-path tests, so those tests can keep validating request shaping without requiring the optional Vertex SDK in the default proxy environment. GitHub is also green now, including the test (3) shard that failed when I requested changes.

Verified locally:

• .venv\\Scripts\\python.exe -m pytest tests/test_vertex_diagnostics.py tests/test_proxy_vertex_native_integration.py tests/test_openai_streaming_backend.py::TestOpenAIStreamingMock::test_litellm_vertex_streaming_preserves_max_tokens_and_vendor_fields tests/test_proxy/test_openai_backend_path.py::test_litellm_vertex_backend_path_preserves_max_tokens_and_vendor_fields -q -> 38 passed, 8 skipped
• .venv\\Scripts\\python.exe -m ruff check ... on the touched Vertex/proxy/test files -> passed
• git diff --check upstream/main...HEAD -> passed

Full GitHub check rollup is green on 5268652007c6068445b1fd251d303d21c9d77249.

[zeroasterisk]

Refreshed and cleanly merged with latest main (c502087d). All Vertex diagnostic suites pass locally (38 passed, 8 skipped) and Ruff is clean. Ready for final review / merge when you have a moment, @JerrettDavis.

[JerrettDavis]

Re-reviewed the exact current head after the main merges. The Vertex failure annotation remains confined to supported failing statuses and preserves successful bodies; the optional-SDK preflight still keys on resolved Vertex aliases and is bypassed only for injected test backends. The prior CI regression fixes remain in both LiteLLM Vertex request-shaping tests. The later Anthropic conflict resolution integrates current main without removing those diagnostics. I also added the missing factual Runtime Rollout Safety section. No new source blocker; fork workflows remain authorization-gated rather than failing.