fix: submit packed headers required by the VA-API driver - #26
Open
maryny4 wants to merge 1 commit into
Open
Conversation
Mesa radeonsi expects the application to submit packed SPS/PPS and slice headers (VAConfigAttribEncPackedHeaders); vlVa parses the packed slice header for nal_ref_idc/nal_unit_type and the slice fields the firmware writes into the final slice NAL. Without them the emitted slices carry a zeroed NAL header byte and reference parameter sets that are never inserted, so the stream never decodes: clients render a permanent white screen while input keeps working (MuNeNiCK#21). Intel iHD composes headers itself, which is why the issue was AMD-only. Query VAConfigAttribEncPackedHeaders at encoder creation and submit packed sequence (SPS+PPS) buffers on IDR frames plus a packed slice header per frame when the driver asks for them. Verified offline on radeonsi (Raphael iGPU, Mesa 26.1.6) with the ignored vaapi_encode_probe test: previously the stream contained only type-0 NAL blobs and zero decodable frames; with packed headers it contains proper IDR/P slices and decodes to the expected solid color. Fixes MuNeNiCK#21 (cherry picked from commit e57ee62)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause of #21
Mesa radeonsi requires the application to submit packed headers (
VAConfigAttribEncPackedHeaders = SEQUENCE | PICTURE | SLICE | …). vlVa parses the packed slice header to learnnal_ref_idc/nal_unit_typeand the slice-header fields the firmware writes into the final slice NAL (picture_h264_enc.c: parseEncSliceParamsH264), and composes SPS/PPS itself from the packed sequence data (radeon_vcn_enc.c: radeon_vcn_enc_encode_h264_header). Without packed headers the emitted slices carry a zeroed NAL header byte and reference parameter sets that are never inserted, so no decoder accepts a single frame: clients render a permanent white screen while input keeps working. Intel iHD composes headers itself, which is why the issue is AMD-only.Offline reproduction on radeonsi (Raphael iGPU, Mesa 26.1.6), no RDP client needed — the new
#[ignore]dvaapi_encode_probetest encodes solid-color frames and dumps the Annex-B stream:00 00 00 01 00 88 80 40 02 …), ffmpeg decodes zero frames;h264_vaapicontrol stream on the same device is byte-for-byte structurally identical except the proper65NAL byte — hardware and driver are fine, the submission is what differs.Fix
Query
VAConfigAttribEncPackedHeadersat encoder creation; when the driver asks for them, submit packed sequence (SPS+PPS) buffers on IDR frames and a packed slice header per frame, built with the existingBitWriterand consistent withgenerate_sps_pps/build_slice_params. On radeonsi the driver then writes SPS/PPS itself (taking only the NAL byte from the packed data) and derives the slice NAL from the parsed header, so the parameter sets and slices always come from one composer. Drivers that do not request packed headers keep the exact previous behavior.After the fix the probe stream contains proper IDR/P slices and decodes to the expected solid color, matching the ffmpeg control output.
Relation to #22
#22 diagnoses the same white screen and confirms it on VCN 3.0 — credit to @stevenwcarter for the finding that the synthesized PPS advertised syntax (transform_8x8) the firmware never emits. This PR takes the complementary contract-level route: with packed headers submitted, radeonsi composes SPS/PPS and slice NALs itself, so the stream is self-consistent without post-hoc NAL repair or vendor-string gating, and the synthesized-PPS mismatches cannot occur by construction. The
encoderselection option proposed separately also gives affected users an explicit software fallback.cargo fmt --check,clippy -- -D warnings,cargo test(both feature sets) pass.Fixes #21