Skip to content

fix(proxy): capture streaming token usage for passthrough providers#1567

Merged
SebConejo merged 1 commit intomainfrom
total-token-display
Apr 14, 2026
Merged

fix(proxy): capture streaming token usage for passthrough providers#1567
SebConejo merged 1 commit intomainfrom
total-token-display

Conversation

@SebConejo
Copy link
Copy Markdown
Member

@SebConejo SebConejo commented Apr 14, 2026

Summary

Fixes two bugs causing streaming responses to log 0 tokens in the dashboard and return 0 tokens to downstream clients (breaking context management in agents like OpenClaw):

  • Passthrough buffer flush: pipeStream() in stream-writer.ts never flushed the passthrough buffer after the stream ended. If the final SSE chunk containing usage didn't end with \n\n, it stayed unparsed and tokens were logged as 0. Added a flush block mirroring the existing transform-path flush.

  • stream_options injection: The proxy didn't inject stream_options.include_usage: true into outgoing requests for OpenAI/OpenRouter. If the calling agent didn't send this field, the provider wouldn't include usage in the stream — so neither Manifest's DB nor the downstream client got token counts. Now injected automatically for streaming requests to passthrough providers.

Transform paths (Google/Anthropic/ChatGPT adapters) were unaffected — they already synthesize usage chunks.

Closes #1477, closes #1502

Test Coverage

  • 4 new tests for passthrough buffer flush (usage capture, non-JSON, [DONE], whitespace-only)
  • 5 new tests for stream_options injection (OpenAI, OpenRouter, non-streaming, non-passthrough, merge with existing options)
  • All 3553 backend tests pass

Pre-Landing Review

No issues found.

Test plan

  • All backend tests pass (3553 tests, 181 suites)
  • New tests cover all new code paths
  • Adversarial review: no actionable findings

🤖 Generated with Claude Code


Summary by cubic

Fixes zero-token logs for streaming responses by capturing usage from passthrough providers. Adds a passthrough buffer flush and injects stream_options.include_usage for openai/openrouter streaming requests to ensure correct token counts in the dashboard and for downstream clients.

  • Bug Fixes
    • Flushes leftover passthrough SSE buffer in pipeStream() to parse final usage chunks without trailing \n\n.
    • Injects stream_options.include_usage: true for streaming requests to openai and openrouter while preserving existing stream_options.
    • Adds tests for buffer flush and injection logic; transform adapters already synthesize usage and are unaffected.

Written for commit 4442ecc. Summary will update on new commits.

Two fixes for streaming responses logging 0 tokens:

1. Flush the passthrough buffer after stream ends so usage from
   the final SSE chunk is captured even without trailing \n\n.

2. Inject stream_options.include_usage for OpenAI/OpenRouter streaming
   requests so providers always send usage data — needed for both
   Manifest DB logging and downstream client context management.

Closes #1477, closes #1502

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.94%. Comparing base (34ab548) to head (4442ecc).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1567   +/-   ##
=======================================
  Coverage   97.94%   97.94%           
=======================================
  Files         118      118           
  Lines        8616     8616           
  Branches     3242     3242           
=======================================
  Hits         8439     8439           
  Misses        175      175           
  Partials        2        2           
Flag Coverage Δ
frontend 97.93% <ø> (ø)
shared 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

@SebConejo SebConejo merged commit 158cbaf into main Apr 14, 2026
15 checks passed
@SebConejo SebConejo deleted the total-token-display branch April 14, 2026 23:54
andychu666 pushed a commit to andychu666/manifest that referenced this pull request Apr 15, 2026
…nfst#1567)

Two fixes for streaming responses logging 0 tokens:

1. Flush the passthrough buffer after stream ends so usage from
   the final SSE chunk is captured even without trailing \n\n.

2. Inject stream_options.include_usage for OpenAI/OpenRouter streaming
   requests so providers always send usage data — needed for both
   Manifest DB logging and downstream client context management.

Closes mnfst#1477, closes mnfst#1502

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
(cherry picked from commit 158cbaf)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming responses from Ollama log 0 tokens in Manifest DB Total token value not shown

1 participant