Skip to content

Fix H.264 codec matching - #931

Merged
ladvoc merged 5 commits into
mainfrom
jacobgelman/clt-2612-h264-codec-matching-bug
Mar 20, 2026
Merged

Fix H.264 codec matching#931
ladvoc merged 5 commits into
mainfrom
jacobgelman/clt-2612-h264-codec-matching-bug

Conversation

@ladvoc

@ladvoc ladvoc commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

Addresses an issue with H.264 streams using packetization-mode=0. This affects tracks published between two cloud regions and all video tracks published by the ESP32 SDK.

Root cause: the platform hardware decoder is not recognized as supporting H.264 streams with packetization-mode=0 because the codec matching logic treats different packetization modes as entirely distinct codecs. As a result, no compatible decoder is found and none is created. Packets are received but have no decoder to process them, so no frames are ever emitted to the user. Other SDKs (Swift, web, etc.) did not have the same issue.

@theomonnom

Copy link
Copy Markdown
Member

Ok to ignore packetization-mode, but I don't think we should ignore profile-level-id

@ladvoc
ladvoc marked this pull request as ready for review March 6, 2026 21:44
@theomonnom

Copy link
Copy Markdown
Member

mmh actually are we sure about that: https://datatracker.ietf.org/doc/html/rfc6184#section-5.4

@davidzhao
davidzhao requested a review from cnderrauber March 9, 2026 23:27
Comment thread webrtc-sys/src/video_decoder_factory.cpp Outdated
@boks1971

Copy link
Copy Markdown

AFAIU, packetization-mode=1 should be able to decode packetization-mode=0 also. So, it should be okay to search for a match as long as the decoder is capable of decoding packetization-mode=1. However, if supported formats does not have packetization-mode, erasing it and searching without it might be risky, i. e. if the supported formats only supports mode=0 (implicit default), searching for match after erasing would match mode=1 also and that would not work. In that case, I am thinking packetization-mode=0 should be used for search.

@boks1971

Copy link
Copy Markdown

AFAIU, packetization-mode=1 should be able to decode packetization-mode=0 also. So, it should be okay to search for a match as long as the decoder is capable of decoding packetization-mode=1. However, if supported formats does not have packetization-mode, erasing it and searching without it might be risky, i. e. if the supported formats only supports mode=0 (implicit default), searching for match after erasing would match mode=1 also and that would not work. In that case, I am thinking packetization-mode=0 should be used for search.

Would be good to test in case there are decoders who support only mode=1 exist, but maybe hard to find.

@ladvoc
ladvoc force-pushed the jacobgelman/clt-2612-h264-codec-matching-bug branch from 6355818 to e72d4a6 Compare March 10, 2026 19:23
@ladvoc

ladvoc commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

I added end-to-end testing for video streaming in #937 to give us more confidence in this change and future changes affecting the video layer. However, as noted in that PR, only VP8 and VP9 can currently be tested in CI, so I will run manually on different platforms to verify the H.264 and H.265 paths.

@chenosaurus

Copy link
Copy Markdown
Contributor

Confirmed ESP32 h264 stream works correctly
also tested Nvidia NVENC h264 stream still works correctly.

@ladvoc
ladvoc force-pushed the jacobgelman/clt-2612-h264-codec-matching-bug branch from e72d4a6 to ed5f1e5 Compare March 20, 2026 17:26
@ladvoc
ladvoc merged commit 6787272 into main Mar 20, 2026
22 checks passed
@ladvoc
ladvoc deleted the jacobgelman/clt-2612-h264-codec-matching-bug branch March 20, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants