Description
On Windows, the log_path option of FFmpeg's libvmaf filter may create a file with an incorrectly encoded filename when the requested filename contains non-ASCII characters.
For example, requesting 你好.json (means “Hello” in Chinese) creates 浣犲ソ.json (mojibake) instead. The encoding mismatch can be demonstrated directly:
>>> "你好".encode("utf-8").decode("cp936")
'浣犲ソ'
The problem affects the filename only. The generated file is a valid VMAF JSON log, and its contents are not garbled or corrupted.
Changing the console code page to 65001 does not help.
Reproduction
ffmpeg -hide_banner -f lavfi -i "testsrc2=size=1280x720:rate=30:duration=1" -filter_complex "[0:v]split=2[dist][ref];[dist][ref]libvmaf=log_fmt=json:log_path='你好.json'" -f null -
After the command completes, the current directory contains a valid JSON log named: 浣犲ソ.json rather than the requested: 你好.json
Other Cases:
测试.json (means "test") -> 娴嬭瘯.json
热力图.json (means "heatmap") -> stderr: could not open file: 鐑姏鍥?json
Comparison with FFmpeg's psnr filter
FFmpeg's built-in psnr filter does not exhibit this problem with the same non-ASCII filename:
ffmpeg -hide_banner -f lavfi -i "testsrc2=size=1280x720:rate=30:duration=1" -filter_complex "[0:v]split=2[dist][ref];[dist][ref]psnr=stats_file='你好.log'" -f null -
This command creates the correctly named file 你好.log
The psnr filter opens its output through FFmpeg's UTF-8-aware file-opening support, whereas the libvmaf output file is opened inside libvmaf. This comparison also shows that the problem is not caused by the shell, the console code page, or FFmpeg's filtergraph parsing of the filename.
Environment
- Windows 10 with default console code page
936
- FFmpeg n8.1 build by BtbN, with libvmaf statically linked into
avfilter-11.dll
- Reproduced with libvmaf commit
0f9912e
Root cause analysis
FFmpeg passes the log_path string to libvmaf, which creates the output file in vmaf_write_output(). The current implementation uses a narrow fopen() call:
int vmaf_write_output(VmafContext *vmaf,
const char *output_path,
enum VmafOutputFormat fmt)
{
FILE *outfile = fopen(output_path, "w");
...
}
On Windows, narrow CRT file APIs do not reliably interpret a UTF-8 pathname when the active ANSI code page is CP936. This explains why the UTF-8 bytes for 你好 are interpreted as CP936 and produce the filename 浣犲ソ.
A possible fix would be to use a UTF-8-aware file-opening path on Windows, for example by converting the UTF-8 pathname to UTF-16 and calling _wfopen(). A Windows regression test should verify the exact filename created for a non-ASCII log_path.
Description
On Windows, the
log_pathoption of FFmpeg'slibvmaffilter may create a file with an incorrectly encoded filename when the requested filename contains non-ASCII characters.For example, requesting
你好.json(means “Hello” in Chinese) creates浣犲ソ.json(mojibake) instead. The encoding mismatch can be demonstrated directly:The problem affects the filename only. The generated file is a valid VMAF JSON log, and its contents are not garbled or corrupted.
Changing the console code page to
65001does not help.Reproduction
After the command completes, the current directory contains a valid JSON log named:
浣犲ソ.jsonrather than the requested:你好.jsonOther Cases:
Comparison with FFmpeg's
psnrfilterFFmpeg's built-in
psnrfilter does not exhibit this problem with the same non-ASCII filename:This command creates the correctly named file
你好.logThe
psnrfilter opens its output through FFmpeg's UTF-8-aware file-opening support, whereas thelibvmafoutput file is opened inside libvmaf. This comparison also shows that the problem is not caused by the shell, the console code page, or FFmpeg's filtergraph parsing of the filename.Environment
936avfilter-11.dll0f9912eRoot cause analysis
FFmpeg passes the
log_pathstring to libvmaf, which creates the output file invmaf_write_output(). The current implementation uses a narrowfopen()call:On Windows, narrow CRT file APIs do not reliably interpret a UTF-8 pathname when the active ANSI code page is CP936. This explains why the UTF-8 bytes for
你好are interpreted as CP936 and produce the filename浣犲ソ.A possible fix would be to use a UTF-8-aware file-opening path on Windows, for example by converting the UTF-8 pathname to UTF-16 and calling
_wfopen(). A Windows regression test should verify the exact filename created for a non-ASCIIlog_path.