CUDA 16bpc motion kernel applies byte stride as uint16_t element stride — motion2 ~-58%, VMAF ~1.5 points low for all >8-bit input

[waveletbeam]

Summary

calculate_motion_score_kernel_16bpc in
libvmaf/src/feature/cuda/integer_motion/motion_score.cu addresses source
rows with uint16_t pointer arithmetic while using src.stride[0], which is
a byte stride (picture_cuda.c: pic->stride[0] = aligned_y << hbd, and
cuMemAllocPitch returns byte pitches). The kernel therefore advances two
image rows per row (and reads past the luma plane for the lower half of the
image). The resulting blurred image is structurally wrong, the temporal SAD
collapses, and the final VMAF score is significantly off for any >8-bit
input. The 8bpc kernel next to it is correct (byte arithmetic on uint8_t*).

Note: this is not the same defect as #1562 (edge-mirror off-by-one, same
file). That one is a boundary artifact on the order of ~1/width affecting
both bit depths; this one is a row-addressing bug that breaks the entire
plane for 16bpc input only. Both exist independently on current master — if
the kernels get rewritten as proposed in #1562, the stride handling below
needs to be part of it.

Affected line (master)

// calculate_motion_score_kernel_16bpc, inner blur loop:
blurred_y += filter_d[yf] * (reinterpret_cast<const uint16_t*>(src.data[0])
    + mirror(y-radius+yf, height) * src.stride[0])[mirror(x-radius+xf, width)];

The CPU reference divides the stride for high bit depth
(integer_motion.c: ref_pic->bpc == 8 ? ref_pic->stride[0] : ref_pic->stride[0] / 2), and the CUDA VIF path converts byte strides to
element strides as well (filter1d.cu) — only the 16bpc motion kernel
does not.

Measurements (isolating the bug)

Setup: FFmpeg (libvmaf/libvmaf_cuda filter), 271 frames 2048x1080, FFV1
10-bit reference vs. AV1 encode, both sides converted identically to
yuv444p16le, model vmaf_v0.6.1. Means from the XML logs; all VIF and ADM
features (all scales) match the CPU reference bit-exactly in every run,
which rules out data upload/conversion issues — only the motion feature
diverges:

run	vmaf	integer_motion2	psnr_y
CUDA @444p16 (before)	83.5315	0.9240	39.6156
CUDA @444p16 (patched)	84.9968	2.2142	39.6156
CPU reference	84.9976	2.2149	39.6156

The per-frame CUDA/CPU motion2 ratio varies
(0.38–0.48), i.e. the error is structural (wrong rows), not a constant
scale factor. With 8-bit yuv420p input the CUDA scores match the CPU
(motion kernel 8bpc is correct), which is presumably why this went
unnoticed. Note the error is content-dependent (motion masking), so it
cannot be corrected by a constant offset.

Side finding: with yuv444p16le input, the CUDA psnr feature reports
psnr_cb/psnr_cr saturated at 108.0 while psnr_y is correct — possibly a
related stride issue in the chroma path of the CUDA PSNR extractor.

Fix (verified)

Advance the row on a byte pointer like the 8bpc kernel does:

-                blurred_y += filter_d[yf] * (reinterpret_cast<const uint16_t*>(src.data[0]) + mirror(y-radius+yf, height) * src.stride[0])[mirror(x-radius+xf, width)];
+                blurred_y += filter_d[yf] * reinterpret_cast<const uint16_t*>(reinterpret_cast<const uint8_t*>(src.data[0]) + mirror(y-radius+yf, height) * src.stride[0])[mirror(x-radius+xf, width)];

After this one-line change, CUDA@yuv444p16le matches the CPU within
0.001 VMAF (see table above). Happy to submit this as a PR if preferred.

Environment

• vmaf master (bug present in GitHub master as of 2026-07-13)
• CUDA 13.0, RTX 4090 (sm_89), Ubuntu 24.04
• Driven via FFmpeg libvmaf_cuda filter (hwupload_cuda + libvmaf_cuda)