266/10bit HDR signalling in mp4 files

[jungleboynx]

I encoded some 10bit HDR/HLG source with -c yuv420_10 and --hdr hlg_2020
I've got a SDR monitor so tone mapping is needed. The raw vvc file plays back with the correct colours with standard mpv (with the native ffmpeg VVC decoder) and another version of mpv that uses the Fraunhofer VVdeC decoder.

After muxing the vvc file into mp4 the standard mpv player now shows the wrong washed out colours (not tone mapped) but the mpv player with Fraunhofer VVdeC plays the correct colours.

I solved the problem by adding the bt2020 signalling information to the mp4 file using
mp4box -add video.mp4#video:colr=nclc,9,18,9 -new video2.mp4

The nclc information is half way down the page:
https://github.com/gpac/gpac/wiki/mp4box-import-opts

MediaInfo now shows:
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant

For standard 8bit/bt709 use
mp4box -add video.mp4#video:colr=nclc,1,1,1 -new video2.mp4

x265 10bit HDR/HLG also has "--atc-sei 18 --pic-struct 0" but I can't find a way to signal these in the mp4 file.

It looks like VVdeC gets the HDR signalling information from the bitstream but the native ffmpeg VVC decoder needs the information in the container. Most players use the native ffmpeg VVC decoder. Both standard mpv and the other version of mpv built with VVdeC now play 266/10bit HDR files with the correct colours in both Window 10 and Linux in Wayland mode. Flatpak mpv v0.39 running under Linux in x11 mode shows the wrong colours.

Playing the 266/10bit/HDR/mp4 file in a Linux VM with standard flatpak mpv shows the wrong colours when running in x11 mode but shows the correct colours in Wayland mode. The other mpv built with VVdeC shows the wrong colours in both x11 and Wayland modes - this is probably a build bug. The latest VirtualBox 7.1.4 r165100 is used for testing running under Windows 10 host.

Windows 11 VM behaviour is different. Standard mpv plays 266/8bit/SDR files ok but 266/10bit/HDR mp4 just shows a black screen. The other mpv built with VVdeC plays 266/10bit/HDR files ok with the correct colours.

Windows 7 VM also plays 266/10bit/HDR mp4 with mpv built with VVdeC with the correct colours. Standard mpv is no longer supported on Windows 7 and doesn't run.

MacOS untested but 10bit HDR files are likely to behave the same as Linux.

I just thought this information might be of interest.

The stuff above is for HDR/HLG. I suppose the question is how do you signal HDR/PQ as a lot more information is needed such as: master-display, max-cll, chromaloc and hdr10-opt - I can't find this information for mp4box.

mp4box has a hdr option which looks promising but I haven't discovered an example of how to use it.

Update 1: HDR isn't supported in x11 and there's no plans to add it. Most Linux distros now use Wayland by default but screen flicker occurs when running Linux in Wayland mode under VirtualBox - there's no screen flicker with x11.

Update 2: Running mpv v0.39 in Windows 11 VM requires mpv.conf setting to "--vo=gpu" - it now tone maps instead of black screen.

Update 3: Running mpv v0.39 in Windows 10 VM shows the wrong colours and no setting in mpv.conf fixes the problem.
mpv built with VVdeC is ok.

[adamjw24]

Thanks for the info. I think it would be more of interest for ffmpeg ppl, so that they could make sure to parse VUI appropriately and expose the information to the player.

[jungleboynx]

mpv with VVdeC was built for x11. I'll rebuild it for Wayland. That'll allow HDR files to be tone mapped for SDR displays like Flatpak mpv.

Xorg might get HDR working in XWayland in future but not x11 itself:
https://gitlab.freedesktop.org/xorg/xserver/-/issues/1037#note_521100

[adamjw24]

Funny, seeing how x11 was designed and developed way before HDR was a thing, and wayland is supposed to be the "new, modern" thing.

Anyway, thanks for the infos. Since I don't see any actions to be taken on our side, I'm closing the issue.

[jungleboynx]

I've reported the 266/HDR 10bit VVC decoder bug used by ffmpeg.