Fix buffered AudioSource.capture_frame hanging when the source drain stalls

[sam-hark]

Before you submit your PR

• I have read the contributing guidelines and validated that this PR will be accepted.
• I have read and followed the principles regarding breaking changes, testing, and code quality.

PR description

Summary

• Bound the buffered NativeAudioSource::capture_frame completion wait (scaled to the queue duration) so a stalled source drain returns a recoverable RtcError instead of hanging forever.
• Split the AudioTrackSource::InternalSource lock so the drain's sink->OnData() no longer runs under the same mutex as the producer — a wedged/slow sink can no longer block capture_frame, while a sink still cannot be freed mid-OnData.
• On a stall/clear_buffer, release the pending completion so the source stays usable; reclaim the completion context on the false return path (previously leaked).

Problem

A buffered native audio source (queue_size_ms > 0, the default used by livekit-agents' _ParticipantAudioOutput) can hang capture_frame forever, silently killing outbound audio while the room still reports Connected. Reported repeatedly and worked around locally, never fixed upstream:

• capture_frame never completes #408 "capture_frame never completes" — reporter pinpointed it: "the buffer never becomes un-full and thus we block forever".
• Capture Frame Consistently Hang Audio Output #420 "Capture Frame Consistently Hang Audio Output" — "highly temperamental to load … the multi-threaded, task-switching nature … GC playing a role".
• Capture frame failed #497 "Capture frame failed" — Python agents: "TTS is lost … on long stories voice just stops".

Root cause

The buffered source is a producer/consumer pair across the FFI:

• Consumer — webrtc-sys/src/audio_track.cpp, AudioTrackSource::InternalSource: a 10 ms RepeatingTask drains the buffer, pushes 10 ms to each sink via sink->OnData(...), and — only when it runs — fires the deferred completion for the chunk that filled the buffer past the notify threshold.
• Producer — libwebrtc/src/native/audio_source.rs, NativeAudioSource::capture_frame: for each chunk it hands C++ a oneshot::Sender and rx.awaits that completion with no timeout.

The drain held mutex_ across the entire OnData loop. So when the drain is slow or CPU-starved, it is almost always holding the lock, and the producer's synchronous FFI capture_frame blocks acquiring that same mutex — often before it even reaches the awaited completion. Either way, capture_frame never returns: _wait_for_playout never completes, generation tasks pile up, and the session goes permanently mute while ICE stays connected.

Fix

• audio_track.cpp / audio_track.h — split the lock. sink_mutex_ guards sinks_ and is held across the OnData loop; buffer_mutex_ guards the buffer and completion state. The producer (capture_frame) only ever takes buffer_mutex_, so a wedged/slow sink can no longer block it — while RemoveSink still serializes against an in-flight OnData, so a sink cannot be freed mid-call. The drain copies the 10 ms frame into a reused scratch buffer (no per-tick allocation).
• audio_source.rs — bound the wait. rx.await → tokio::time::timeout(...), scaled to the queue duration (5 × queue_ms, floored at 1 s) so legitimate backpressure never trips it. On timeout, clear_buffer() and return RtcError { InvalidState }.
• Keep the source usable + no leak. clear_buffer() now also releases any pending completion (freeing its context and clearing the slot), so a fresh capture_frame succeeds after a stall clears rather than erroring forever. The false-return path now reclaims the boxed Sender it previously leaked.

The timeout and the lock split are coupled: with the drain holding the mutex across OnData, the producer blocks in the FFI before the await, so bounding the await alone would be dead code in exactly the scenario it targets.

Breaking changes

None. capture_frame gains a recoverable RtcError for the stall case, replacing an unobservable permanent hang; callers already handle capture_frame errors.

MSRV

Unchanged.

Testing

libwebrtc/src/native/drain_stall_tests.rs, exercised by the standard cargo test:

• capture_frame_recovers_when_drain_stalls — attaches a sink whose OnData blocks (a deterministic stalled consumer) to a real AudioTrackSource, feeds capture_frame, asserts it returns the drain-stall RtcError (bounded so a wiring regression fails cleanly rather than hanging CI), then — after the stall clears — asserts a fresh capture_frame succeeds (the source is not poisoned). Gated to Linux: it needs a real audio track, and on macOS/Windows that brings up the platform AudioDeviceModule, whose real-time thread aborts the process when the blocking sink stalls it. The fix is platform-independent.
• capture_frame_completes_under_normal_backpressure — runs on every platform (no track/sink); with the drain running, deferred captures complete Ok, confirming the timeout does not false-trip on legitimate backpressure. Uses a large queue so the assertion is not coupled to the production floor.

Verified on a real webrtc build: stock hangs; timeout-without-lock-split still hangs (producer wedged on the mutex); the combined fix returns an error in ~1 s and the source recovers; the full libwebrtc lib suite passes.

Async

The bounded wait is the fix itself; the test also uses tokio::time::timeout to detect a hang, mirroring the existing renegotiation_does_not_deadlock test, and subscribes to a channel signalled the instant the drain enters the sink rather than sleeping for it.

Out of scope (deferred)

capture_frame remains a synchronous FFI call that can block its caller's task while the drain catches up; a fuller change might make the enqueue non-blocking (or run it off the async runtime). This PR keeps that shape and only ensures the wait is bounded and the lock no longer couples the producer to a wedged sink.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[xianshijing-lk]

Curiously, in what setup or workload did you actually hit this timeout? My read is that normal Python/agents usage should await capture_frame() and therefore use it as backpressure, so this would only happen if the native webrtc drain stops making progress. Not sure how that could happen.

Was your repro tied to long TTS/audio bursts, high process load, CPU starvation, shutdown, or other workload sensitive conditions?

[sam-hark]

My repro was tied to high process load mostly during long TTS playout, but it has also been very intermittent and difficult to pin down. To be honest, I can't claim this is definitely the root cause of audio stalls I've seen, but it's my leading cause.

It appears that under load the drain task stops getting scheduled, so the queue stays full and the capture_frame await never returns.

[xianshijing-lk]

should we move this scratch_.resize(samples10ms); to the constructor ? I would think we don't need to call resize() for every tick.

[sam-hark]

Good point, I updated

[xianshijing-lk]

This is a known problem and not related to your change, but probably we shouldn't call on_complete_() under the buffer_mutex_ lock. Can you change the code to

void (complete)(const SourceContext) = nullptr;
const SourceContext* complete_ctx = nullptr;
{
webrtc::MutexLock lock(&buffer_mutex_);
buffer_.clear();

  complete = on_complete_;
  complete_ctx = capture_userdata_;
  on_complete_ = nullptr;
  capture_userdata_ = nullptr;
}

if (complete)
  complete(complete_ctx);

}

[sam-hark]

Yes, I updated

[xianshijing-lk]

Is sink->OnData() the main problem that could block forever ?

WebRTC’s send-side OnData is intended to be fast, this OnData just copies the frame and posts encoding work to an encoder task queue. It shouldn't block for long

[sam-hark]

Likely not, I could remove this from the PR if preferred. This is just trying to enforce that the producer shouldn't be able to block the sink, and it enables adding a deterministic unit test so that the timeout fix can be directly tested.