Skip to content

Retire the MoQ audio track on interruption - #5716

Open
kixelated wants to merge 2 commits into
pipecat-ai:mainfrom
kixelated:moq-interrupt-audio-track
Open

Retire the MoQ audio track on interruption#5716
kixelated wants to merge 2 commits into
pipecat-ai:mainfrom
kixelated:moq-interrupt-audio-track

Conversation

@kixelated

Copy link
Copy Markdown
Contributor

Summary

MOQOutputTransport retires its audio track when an interruption catches audio still to be played, publishing a uniquely generation-named successor (bot-audio, bot-audio-1, ...) in its place. This replaces the pacing-clock re-anchor.

The bot writes TTS up to audio_out_max_buffer_ms ahead of real-time, so at barge-in seconds of the abandoned utterance have already left the process. Nothing on the bot side could retract it, so discarding it relied on an out-of-band signal: the bot published RTVI user-started-speaking on the transcript track and the browser flushed its buffer. The two tracks have no ordering relative to each other, and the transcript is subscribed at priority 0 against audio's 80, so the flush signal is scheduled behind exactly the data it exists to flush.

Retiring the track makes the discard structural instead. AudioProducer.finish() drops the producer's catalog rendition, and the successor publishes its own, so the catalog always advertises exactly one audio track and subscribers follow the swap through normal rendition selection.

When the swap happens

Interruptions are broadcast on every user turn, not only on barge-in, so an unconditional swap would cost every subscriber a catalog update and a resubscribe per turn. A swap happens only when the pacing clock's lead over wall-clock exceeds AUDIO_IN_FLIGHT_FLOOR_S (0.3s).

Timestamps

PTS is not per-track. Every track stamps against one broadcast-wide monotonic origin, matching the browser side's performance.now() convention, so tracks stay mutually synchronizable. A successor re-anchors into that timeline at the position it starts playing from rather than continuing from the write-ahead frontier or resetting to zero.

A track takes its epoch at its first write, not when it opens. Only a track's first stamp reaches the wire as given: moq-audio's encode producer takes it as that track's epoch and derives every later PTS from the running sample count. A new producer resets only its group sequence to zero.

Ordering

  • process_frame calls super() first so the base class drains the audio queue before the track is retired.
  • publish_audio captures its producer up front and drops the chunk if the track was retired while it sat in the pacing sleep.

Test plan

TestMOQAudioTrackRestart and TestMOQAudioTrackRestartLive in tests/test_moq_transport.py cover:

  • catalog replacement (finish + uniquely named successor; live catalog advertises exactly one audio track)
  • continuous PTS on the broadcast timeline (not zero, not the write-ahead frontier)
  • stale paced writes dropped
  • group zero on every replacement track (live AudioProducer)

uv run pytest tests/test_moq_transport.py passes 63 tests with the moq extra installed. ruff check and ruff format --check clean.

This supersedes #5443, rebased onto current main with live catalog and group-zero coverage.

(written by grok-4.6)

kixelated and others added 2 commits September 11, 2026 23:46
The bot paces TTS ahead of real-time, so when the user barges in the rest
of the utterance is already encoded and in flight. Retiring the track is
what discards it: finish() drops the catalog rendition, a uniquely
generation-named successor publishes its own, and catalog-aware players
follow the swap.

PTS stays on one broadcast-wide timeline. A replacement does not reset
the origin to zero; the successor re-anchors at its first write so the
new utterance plays immediately instead of waiting out the abandoned
write-ahead. Only the new producer's group sequence starts at zero.

publish_audio captures its producer before the pacing wait and drops the
chunk if rotation replaced the track while it was asleep.

Co-Authored-By: grok-4.6 <noreply@x.ai>
Co-Authored-By: grok-4.6 <noreply@x.ai>
@codecov

codecov Bot commented Sep 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 37 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pipecat/transports/moq/transport.py 0.00% 37 Missing ⚠️
Files with missing lines Coverage Δ
src/pipecat/transports/moq/transport.py 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant