Skip to content

Attach per-turn call audio to the Langfuse trace - #240

Closed
jamsea wants to merge 4 commits into
mainfrom
jh/langfuse-audio
Closed

jamsea wants to merge 4 commits into
mainfrom
jh/langfuse-audio

Conversation

@jamsea

@jamsea jamsea commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

What

The Langfuse example now attaches the call audio to the trace, at two levels:

  • The whole call, on the trace root. Stereo, user left and bot right, so an interruption
    reads as overlap.
  • Each turn, on that turn's turn span. The user's speech is filed as the span's
    input and the bot's reply as its output, so clicking a turn plays just that turn
    and you can hear each side separately.

Opt-in: set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_HOST and the audio
appears. Leave them unset and you get traces exactly as before.

How it works

Langfuse media does not travel over OTLP. Audio is uploaded through
POST /api/public/media → presigned PUTPATCH /api/public/media/{id}, and linked to a
trace, or to one observation inside it, by id.

The useful property is that Langfuse renders a player from that link alone, with nothing
written into the span payload. So a clip only needs the trace id, plus the turn's span id for
per-turn audio, and both outlive the spans themselves (TurnTraceObserver.get_turn_context()
retains them). That means:

  • No span is ever mutated, so Langfuse's immutable-trace rule never comes into play.
  • Uploading happens after the pipeline has shut down, so a slow upload never delays teardown
    or holds a call open.
  • Everything uses public observer API: turn_tracking_observer to number the turns and
    get_turn_context() to find each turn's span.

Per-turn uploads are capped at max_turn_clips (40 by default) because each clip costs two
calls against Langfuse's general API rate limit (30/min on Hobby, 100/min on Core). A 429 is
retried once using Retry-After.

No new dependencies: aiohttp already comes with pipecat-ai.

Drive-by bug fix

env.example set OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64>" with a literal
space. Header values in that variable are URL encoded per the OTLP spec, and the OTel SDK
discards a value it cannot parse:

'Authorization=Basic%20QUJDOmRlZg==' -> {'authorization': 'Basic QUJDOmRlZg=='}
'Authorization=Basic QUJDOmRlZg=='   -> {}

So anyone who copied env.example verbatim sent no auth header and got
Failed to export span batch code: 401 with zero traces. The README already showed %20;
env.example disagreed. Happy to split this into its own PR if you would rather not have a
bug fix riding along with a feature.

Also included

  • An eval transport entry so pipecat eval run can drive this bot. Imported lazily, since
    the harness is an optional extra.
  • A startup warning when LANGFUSE_HOST and OTEL_EXPORTER_OTLP_ENDPOINT name different
    regions. Mixing EU and US puts the audio in one project and the trace in another, with no
    error anywhere.

jamsea added 2 commits August 10, 2026 17:01
The Langfuse example produced traces but no audio, so you could read what the bot
decided without hearing what the call sounded like. The LangSmith example already
records the call and attaches it; this brings Langfuse to parity.

Langfuse media does not travel over OTLP. The WAV is uploaded through the media
REST API and referenced from the trace by a token. Langfuse also treats a
persisted trace as immutable, so the token has to be written onto the conversation
span while that span is still open, which is why finalize() runs before
worker.cancel() rather than during cleanup.

Also fixes OTEL_EXPORTER_OTLP_HEADERS in env.example, which used a literal space
after "Basic". The OTLP spec requires that value URL encoded, and the OTel SDK
discards a header it cannot parse, so every span export failed with a 401 for
anyone who copied the file. The README already showed %20; env.example did not.

Adds an "eval" transport entry so `pipecat eval run` can drive the bot.
One recording for a whole call is hard to navigate: to hear what happened on a
turn you had to scrub. Now each turn span carries its own audio, with the user's
speech filed as the span's input and the bot's reply as its output, so clicking a
turn plays just that turn and each side is separate. The whole-call recording is
still attached to the trace root.

This also removes the trickiest part of the previous design. Langfuse renders a
player from the media link alone, with nothing written into the span payload, so
a clip only needs the trace id and the turn's span id. Those ids outlive the
spans, which means uploads no longer have to beat the conversation span closing
and can run after the pipeline has shut down. No span is mutated, and everything
uses public observer API.

Per-turn uploads are capped (max_turn_clips, 40 by default) because each clip
costs two calls against Langfuse's general API rate limit, and a 429 is retried
once using Retry-After.
@jamsea jamsea changed the title Attach the call recording to the Langfuse trace Attach per-turn call audio to the Langfuse trace Aug 11, 2026
Replace the hand-rolled aiohttp calls (basic auth, media POST, confirmation
PATCH, 429 retry loop) with the langfuse package's generated AsyncLangfuseAPI
client, which does auth and Retry-After backoff itself. The presigned PUT to
object storage stays manual since the SDK only does it in private code.

The full Langfuse() client is deliberately not used: it starts its own OTel
tracer provider, which would fight with Pipecat's setup_tracing().
The 469-line langfuse_media.py moved into pipecat as
pipecat.utils.tracing.langfuse (pipecat-ai/pipecat#5285), so the example is
now just the wiring: from_env(), attach(), stop_and_collect(), upload().

That module is not in a release yet, so pipecat installs from the PR branch.
Switch back to a version pin once it ships.
@jamsea
jamsea requested review from markbackman and a lite review from Copilot August 11, 2026 08:00
@jamsea
jamsea marked this pull request as ready for review August 11, 2026 08:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the Langfuse OpenTelemetry example to upload and attach call audio recordings to Langfuse traces (whole-call on the trace root and per-turn clips on turn spans), alongside documentation updates and an env.example fix for OTLP header encoding.

Changes:

  • Add audio recording + Langfuse media upload wiring to the Langfuse demo bot (plus an eval transport option).
  • Expand Langfuse README with recording setup, behavior notes, and eval instructions.
  • Fix OTEL_EXPORTER_OTLP_HEADERS formatting in env.example to use %20 instead of a literal space.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
open-telemetry/README.md Updates the demos table to mention Langfuse audio playback support.
open-telemetry/langfuse/README.md Adds end-user documentation for attaching whole-call and per-turn audio, plus eval guidance and troubleshooting.
open-telemetry/langfuse/pyproject.toml Switches pipecat-ai dependency to a PR branch to access LangfuseRecordingUploader.
open-telemetry/langfuse/env.example Fixes OTLP header encoding and documents optional Langfuse REST credentials for audio upload.
open-telemetry/langfuse/bot.py Adds AudioBufferProcessor + uploader integration and an eval transport option.
Suppressed comments (3)

open-telemetry/langfuse/bot.py:196

  • If recording is made conditional on uploader, the else: await audiobuffer.stop_recording() path can end up calling stop_recording() without a matching start_recording(), and still does unnecessary work when audio is disabled. Remove the else branch and only stop/collect when uploader is enabled.
        if uploader:
            await uploader.stop_and_collect(audiobuffer, worker)
        else:
            await audiobuffer.stop_recording()

open-telemetry/langfuse/README.md:131

  • In the README snippet, uploader can be None (keys unset), but await uploader.stop_and_collect(...) is still called unconditionally, which would fail if someone follows the “leave them unset” guidance. Guard this call with if uploader: in the example.
    # Collect the audio while the pipeline is still up; upload after it shuts down.
    await uploader.stop_and_collect(audiobuffer, worker)
    await worker.cancel()

open-telemetry/langfuse/README.md:134

  • Similarly, the README snippet uploads unconditionally (await uploader.upload(worker)), but uploader may be None. Wrap the upload in if uploader: so the snippet can be copy/pasted safely when audio is not configured.
# After the runner returns:
await uploader.upload(worker)

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 180 to +183
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
await audiobuffer.start_recording()
Comment on lines +106 to +107
from pipecat.utils.tracing.langfuse import LangfuseRecordingUploader

Comment on lines +123 to +125
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
await audiobuffer.start_recording()
Comment on lines +7 to +10
# LangfuseRecordingUploader (pipecat.utils.tracing.langfuse) is not released yet, so
# install pipecat from the PR branch (pipecat-ai/pipecat#5285). Switch back to a
# version pin once it ships.
"pipecat-ai[daily,webrtc,websocket,silero,cartesia,deepgram,openai,tracing,runner] @ git+https://github.com/pipecat-ai/pipecat.git@jh/turn-audio-turn-number",
@markbackman

Copy link
Copy Markdown
Contributor

Closing this in favor of the the approach you posted to Pipecat. In #242, I use your approach for the LangfuseRecorder, which is the application code to maintain. 242 will use the two Pipecat utils that will be ready in the next release, so you can only run if you point to a branch of Pipecat, at the moment.

Thanks for doing the ground work on all of this. It made pulling core items into Pipecat much easier 🙇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants