[Core] EJB: revised elastic jitter buffer — reduces per-call max delay ~2× in canary A/B - #3118
Open
jchavanton wants to merge 4 commits into
Open
[Core] EJB: revised elastic jitter buffer — reduces per-call max delay ~2× in canary A/B#3118jchavanton wants to merge 4 commits into
jchavanton wants to merge 4 commits into
Conversation
Introduce packet_stats_t (in_count, in_plc, in_rx, count) and packet_stats_io_info_t (codec/ssrc/addr/callid per direction) on the session. Wire counter increments from the RTP path, the bridge, and the state machine; expose a per-leg JB stats export hook for v2 hangup metrics. Exposes the in_rx counter (packets received from peer, pre-JB) that the damage model joins against bridge/JB writes to compute silence loss and concealment rates. Touches: switch_core.h, switch_core_pvt.h, switch_core_media.h, switch_frame.h (received_ts), switch_rtp.h, switch_core_media.c, switch_core_session.c, switch_core_state_machine.c, switch_ivr_bridge.c, switch_rtp.c.
Track FEC, PLC, and frame counters on the opus decoder. Export them as channel variables (opus_decoder_frames/plc/fec) at decoder teardown so v2 hangup metrics can record the codec-truth concealment count, independent of any bridge-side SFF_PLC accounting. Add gap-tolerant run-length classification: PLC frames in runs ≤3 (≤60ms) are inaudible spectral interpolation; runs >3 degenerate into audible comfort noise. Split exported as plc_short / plc_long / plc_long_starts. A run is gap-tolerant within 200ms so glitchy PLC-real-PLC patterns still count as one audible event.
Comprehensive rework of the elastic jitter buffer covering tuning, metrics, and a new arrival-time jitter estimator: - Two-tier EWMA+EWMV jitter estimator: fast (~320ms) and slow (~10s) tiers exported as jitter_mean_ms/sigma_ms (+ _slow_ms variants). Slow tier and clock-drift state survive switch_jb_reset. - Clock-drift detection from RTP timestamp sampling, logged as ppm. - Categorized reset accounting: reset_too_big, reset_too_expanded (with dropped-packet count), reset_missing_frames, plus buffering_skip and expand_ms. - Expand floor and "keep elastic on miss" so brief gaps don't collapse the EJB to non-elastic mode. - Export switch_jb_is_elastic() and switch_jb_export_stats() so hangup metrics can pull JB state per leg. - Headline counters surfaced via switch_core_media_export_jb_stats feed the v2 damage model (silence loss vs concealment). Includes earlier statistics fixes, duplicate overflow-block removal, small tuning adjustments, and follow-up EJB auto-disable fix.
…ed_pkt) Expose the RTCP receiver-report loss counters as channel variables so hangup metrics and applications can measure true inbound packet loss (carrier→FS) independently of JB behavior: rtp_audio_in_cum_lost - cumulative packets never received rtp_audio_in_expected_pkt - packets expected (high_ext_seq - base + 1) Ratio cum_lost / expected_pkt is the honest inbound loss rate. Prior JB damage models could only observe silence and concealment; adding these lets us separate carrier-side loss from JB-side loss and stop misattributing the two. Populated only when rtcp-audio-interval-msec is set on the sip profile (without RTCP the sequence-tracking path in rtcp_stats early-returns). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework of the elastic jitter buffer and the surrounding instrumentation
needed to distinguish JB-induced audio loss from carrier-side loss.
Restructured from prior JB-only history into 4 logical commits:
[Core] EJB: add per-leg RTP packet statistics —
packet_stats_ton the session plus counters wired through the RTPread path, bridge, and state machine. Adds the
in_rxcounter(packets received pre-JB) that the damage model joins against
bridge/JB writes.
[Codec] Opus: per-call PLC/FEC stats and run-length classification —
Codec-truth PLC/FEC counters exported as
opus_decoder_frames/plc/fec. Gap-tolerant run-lengthclassification splits PLC into
plc_short(≤60ms, inaudible) andplc_long(>60ms, audible comfort noise).[Core] EJB: elastic jitter buffer overhaul — the core rework:
exported as
jitter_mean_ms,sigma_ms, and_slow_msvariants.switch_jb_reset), logged as ppm.reset_too_big,reset_too_expanded,reset_missing_frames, plusbuffering_skipandexpand_ms.EJB on brief gaps.
switch_jb_is_elastic()andswitch_jb_export_stats()forhangup-time per-leg export.
[Core] Media: export inbound sequence-loss counters —
rtp_audio_in_cum_lostandrtp_audio_in_expected_pktchannelvariables from the RTCP receiver-report path, so
cum_lost / expected_pktgives the true inbound loss rate independent of JBbehavior. Populated only when
rtcp-audio-interval-msecis set.Damage model
Together these expose enough state to separate three sources of audio
loss on a call:
rtp_audio_in_cum_lost / expected_pktmax(0, in_rx − jb_out − plc)plc_short) and audible(
plc_long) via opus run-length classificationTesting
Validated in canary via A/B testing (EJB on vs off), using the counters
introduced in commits 1–4 to compare per-leg audio quality on matched
call volumes. Key findings:
to accumulate over the call, resulting in roughly 2× the per-call max
delay compared with EJB (bucket-level p95: EJB-off ~1640ms vs EJB-on
~520ms).
is
silence + plc_long, wheresilence = max(0, in_rx − jb_out − plc)captures JB-side loss andplc_long(>60ms runs) is theaudible portion of concealment.
plc_shortis inaudible spectralinterpolation and should not be counted as damage.
cum_lost / expected_pktcleanly separatescarrier-side loss (upstream of FS) from JB-side loss, so
regressions can be attributed correctly.
normal when the categorized
reset_*counters are all zero; thecategorized counters isolate pathological cases (
reset_too_big,reset_too_expanded,reset_missing_frames).