Retire the MoQ audio track on interruption - #5716
Open
kixelated wants to merge 2 commits into
Open
Conversation
The bot paces TTS ahead of real-time, so when the user barges in the rest of the utterance is already encoded and in flight. Retiring the track is what discards it: finish() drops the catalog rendition, a uniquely generation-named successor publishes its own, and catalog-aware players follow the swap. PTS stays on one broadcast-wide timeline. A replacement does not reset the origin to zero; the successor re-anchors at its first write so the new utterance plays immediately instead of waiting out the abandoned write-ahead. Only the new producer's group sequence starts at zero. publish_audio captures its producer before the pacing wait and drops the chunk if rotation replaced the track while it was asleep. Co-Authored-By: grok-4.6 <noreply@x.ai>
Co-Authored-By: grok-4.6 <noreply@x.ai>
Codecov Report❌ Patch coverage is
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MOQOutputTransportretires its audio track when an interruption catches audio still to be played, publishing a uniquely generation-named successor (bot-audio,bot-audio-1, ...) in its place. This replaces the pacing-clock re-anchor.The bot writes TTS up to
audio_out_max_buffer_msahead of real-time, so at barge-in seconds of the abandoned utterance have already left the process. Nothing on the bot side could retract it, so discarding it relied on an out-of-band signal: the bot published RTVIuser-started-speakingon the transcript track and the browser flushed its buffer. The two tracks have no ordering relative to each other, and the transcript is subscribed at priority 0 against audio's 80, so the flush signal is scheduled behind exactly the data it exists to flush.Retiring the track makes the discard structural instead.
AudioProducer.finish()drops the producer's catalog rendition, and the successor publishes its own, so the catalog always advertises exactly one audio track and subscribers follow the swap through normal rendition selection.When the swap happens
Interruptions are broadcast on every user turn, not only on barge-in, so an unconditional swap would cost every subscriber a catalog update and a resubscribe per turn. A swap happens only when the pacing clock's lead over wall-clock exceeds
AUDIO_IN_FLIGHT_FLOOR_S(0.3s).Timestamps
PTS is not per-track. Every track stamps against one broadcast-wide monotonic origin, matching the browser side's
performance.now()convention, so tracks stay mutually synchronizable. A successor re-anchors into that timeline at the position it starts playing from rather than continuing from the write-ahead frontier or resetting to zero.A track takes its epoch at its first write, not when it opens. Only a track's first stamp reaches the wire as given:
moq-audio's encode producer takes it as that track's epoch and derives every later PTS from the running sample count. A new producer resets only its group sequence to zero.Ordering
process_framecallssuper()first so the base class drains the audio queue before the track is retired.publish_audiocaptures its producer up front and drops the chunk if the track was retired while it sat in the pacing sleep.Test plan
TestMOQAudioTrackRestartandTestMOQAudioTrackRestartLiveintests/test_moq_transport.pycover:AudioProducer)uv run pytest tests/test_moq_transport.pypasses 63 tests with themoqextra installed.ruff checkandruff format --checkclean.This supersedes #5443, rebased onto current main with live catalog and group-zero coverage.
(written by grok-4.6)