Skip to content

fix(tracing): never let span export block the event loop or the run path - #199

Merged
xfgong merged 2 commits into
mainfrom
2026-07-06-nonblocking-trace-flush
Jul 6, 2026
Merged

fix(tracing): never let span export block the event loop or the run path#199
xfgong merged 2 commits into
mainfrom
2026-07-06-nonblocking-trace-flush

Conversation

@xfgong

@xfgong xfgong commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Problem

Diagnosed in cubebox: every chat turn's `done` event lagged the final `usage`
event by 5–20s. Timing probes put 8.65s inside the exit of
`async with trace(tracer, agent)`
— span export was gating a user-visible
response, and worse, freezing the whole event loop while it ran:

  1. `Tracer.force_flush` (and `Meter.force_flush`) are `async def` but called
    the provider's synchronous `force_flush` inline. Awaiting them parked a
    blocking flush on the loop thread: every concurrent request in the host
    process stalls until the exporters drain. Against a remote/backlogged OTLP
    collector that's seconds per run.
  2. `trace()` awaits the flush task on exit by design ("exit means exported").
    Right default for scripts; wrong for serving paths where a caller is
    blocked on the `async with` body's completion.

Fix

  • `Tracer.force_flush` / `Meter.force_flush` now run the provider flush via
    `asyncio.to_thread` — awaiting them never stalls the loop.
  • `trace()` gains `flush: Literal["await", "background"] = "await"`.
    • `"await"` (default): unchanged semantics.
    • `"background"`: detach stays synchronous (listeners off on the next
      line), export runs as a supervised background task. The tracer holds
      strong refs (loop refs are weak) and `shutdown()` settles pending
      flushes, so spans still export on clean shutdown. Supervisor logs
      failures, matching `trace()`'s swallow-and-log contract.

Tests

`tests/tracing/test_nonblocking_flush.py`:

  • provider flush runs off the loop thread; a 0.5s export no longer freezes a
    concurrently ticking task
  • `flush="background"` exits the block immediately, tracks the task, and
    `shutdown()` settles it; failures are logged
  • regression pin: default mode still blocks until export completes

Full suite: 1842 passed, 1 skipped. ruff + format + mypy clean.

🤖 Generated with Claude Code

Two fixes for host-visible latency caused by span export:

1. Tracer.force_flush / Meter.force_flush wrapped the provider's
   synchronous force_flush directly in an async def — awaiting it
   parked the sync flush ON the event loop thread, freezing every
   concurrent task in the host process until the exporters drained
   (seconds against a remote/backlogged OTLP collector). Both now run
   the provider flush via asyncio.to_thread.

2. trace() gains flush="await"|"background". The default keeps
   today's exit-means-exported guarantee. "background" detaches
   synchronously but supervises the export as a tracked background
   task — for serving paths where a caller is waiting on the block
   (measured 8.6s added to chat-turn completion in cubebox before
   this). The tracer holds strong refs to pending flushes and
   shutdown() settles them, so spans still export on clean shutdown;
   supervisor logs failures per this helper's swallow-and-log
   contract.
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

codecov/patch flagged the meter's to_thread flush lines — no existing
meter test exercised force_flush. Mirror the tracer's off-loop thread
assertion.
@xfgong
xfgong merged commit 94d0ca7 into main Jul 6, 2026
14 checks passed
xfgong added a commit to cubeplexai/cubeplex that referenced this pull request Jul 6, 2026
cubeplexai/cubepi#199 merged; move the pin from the branch commit to
the mainline SHA (same code, rebase-merged).
@xfgong
xfgong deleted the 2026-07-06-nonblocking-trace-flush branch July 6, 2026 13:32
xfgong added a commit to cubeplexai/cubeplex that referenced this pull request Jul 6, 2026
cubeplexai/cubepi#199 merged; move the pin from the branch commit to
the mainline SHA (same code, rebase-merged).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant