fix(proxy): estimate streamed output from content

[JerrettDavis]

Description

Replace the streaming fallback that estimates completion tokens from the entire SSE envelope with provider-aware counting of the generated content itself. This prevents event names, JSON metadata, identifiers, and chunk framing from distorting model-output accounting when an upstream omits usage.

Closes #2712

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Performance improvement
• Code refactoring (no functional changes)

Changes Made

• Extract generated text, reasoning, and tool-argument deltas from Anthropic, OpenAI Chat/Responses, and Gemini SSE events.
• Reassemble provider deltas before tokenization so one-character chunks do not create artificial token boundaries.
• Preserve provider-reported usage as authoritative and use content estimation only when usage is absent.
• Replace the estimated_bytes accounting tag with estimated_content and add cross-provider regression coverage.

Testing

• Unit tests pass (pytest)
• Linting passes (ruff check .)
• Type checking passes (mypy headroom)
• New tests added for new functionality
• Manual testing performed

Test Output

pytest -q tests/test_proxy_streaming_request_logger.py tests/test_sse_utf8_split.py tests/test_sse_thinking_blocks.py tests/test_backend_streaming_cache_metrics.py tests/test_backend_bugs.py
76 passed in 1.51s

pytest -q tests/test_proxy_streaming_request_logger.py
15 passed in 0.21s

ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py
All checks passed!

ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py
2 files already formatted

mypy headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file

Real Behavior Proof

• Environment: macOS, Python 3.13, current main at bbe901319.
• Exact command / steps: parse an Anthropic SSE delta containing Hello there plus 80 KB of envelope metadata, then finalize a no-usage stream with 500 KB of framing and eight one-character content deltas.
• Observed result: only generated content is retained for estimation; envelope padding has no effect, and character deltas are joined before tokenization.
• Not tested: live provider streams without usage because no provider credentials are required or available for this deterministic accounting path.

Runtime Rollout Safety

• Rollout-managed feature(s): none.
• Minimum rollout channel: stable.
• Stable/default behavior changed: only fallback output accounting when the provider omits usage; streamed response bytes are unchanged.
• Kill switch / disable path: provider-reported output usage continues to bypass estimation entirely.
• Unsafe override required: no.
• Qualification impact: adds provider-shape and pathological chunk-boundary tests to the existing streaming finalizer suite.
• Rollback path: fix forward in generated-content extraction; do not restore raw-envelope byte accounting.

Review Readiness

• I have performed a self-review
• This PR is ready for human review

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

Not applicable; this changes streaming usage accounting only.

Additional Notes

The prior bytes // 40 heuristic counted the protocol envelope rather than model output and could be arbitrarily distorted by metadata or chunk framing. The new fallback is bounded to content the provider actually generated.

[github-actions]

PR governance

This PR follows the template and is marked ready for human review.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 67.16418% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
headroom/proxy/handlers/streaming.py	67.16%	10 Missing and 12 partials ⚠️

📢 Thoughts on this report? Let us know!