mpv
mpv copied to clipboard
mpv-player
vo_gpu_next: HDR10 passthrough alters the black point
Important Information
- mpv version:
mpv 0.34.0-438-g0f30b0757f Copyright © 2000-2022 mpv/MPlayer/mplayer2 projects
built on Tue Aug 30 05:39:09 2022
FFmpeg library versions:
libavutil 57.28.100
libavcodec 59.37.100
libavformat 59.27.100
libswscale 6.7.100
libavfilter 8.44.100
libswresample 4.7.100
FFmpeg version: n5.1
- Linux Distribution and Version: Arch Linux
- Source of the mpv binary: mpv-git PKGBUILD
- Window Manager and version: Sway 1.7, wlroots 0.15.1
- GPU driver and version: amdgpu, Linux 5.19.4
- libplacebo version: libplacebo v4.208.0.99.g0ce3fa4-1 (API v217)
Reproduction steps
- Run mpv with
mpv --no-config --vo=gpu-next --target-trc=pq --target-prim=bt.2020 --target-peak=10000 --screenshot-format=png --screenshot-high-bit-depth=no test.hevc - Save screenshot
- Compare with
--vo=gpu
Expected behavior
The image should look identical to the input frame.
Actual behavior
The output image has an altered black point.
Only happens using vo=gpu-next
Log file
Log with vo=gpu-next ilovempv.log
Sample files
Rename to test.hevc: https://0x0.st/opu_.hevc
vo=gpu-next screenshot: https://0x0.st/opuf.png
vo=gpu screenshot: https://0x0.st/opuO.png
I think this is technically intended behavior. The justifications behind the change of the default output black point still apply, no? On an OLED display, the un-altered image would look too bright.
In general, the current behavior relies on the assumption that, in the absence of information about the display capabilities (e.g. from EDID), we send a signal that extends to true black and rely on the display to tone-map this to the actual black point of the device.
On an OLED display, the un-altered image would look too bright.
Yes, but wouldn't it be intended to be that luminance as coded in the content? PQ is absolute after all. In this case the content supposedly has a 0.0050 nits black point, but in reality it is probably lower. So remapping it causes some crush.
I'm not sure if any display would remap the black point at all, it would have to be tested.
In this case the content supposedly has a 0.0050 nits black point, but in reality it is probably lower. So remapping it causes some crush.
This is interesting, if it contains code points below the mastering display black point then we would indeed expect massive problems. Maybe we should ignore the mastering metadata black point altogether and assume all PQ content goes down to true black?
Edit: Actually, what we should do is extend our peak detection shader to also detect the black point. And actually, we should probably do a full histogram, like madVR does. That would be interesting information to play with.
I'm not sure if any display would remap the black point at all, it would have to be tested.
No displays map the black point down, which is exactly what prompted this change in https://github.com/mpv-player/mpv/issues/10558
This is interesting, if it contains code points below the mastering display black point then we would indeed expect massive problems. Maybe we should ignore the mastering metadata black point altogether and assume all PQ content goes down to true black?
The mastering metadata min luminance definitely can't be trusted. This title was released by different distributors with different metadata, yet the black floor is the same. So this makes things confusing.
I have yet to actually look at the coded values.
This is interesting, if it contains code points below the mastering display black point then we would indeed expect massive problems. Maybe we should ignore the mastering metadata black point altogether and assume all PQ content goes down to true black?
I can confirm that the raw black point for pretty much all HDR content I have watched always goes down to true black (0). Min Mastering Luminance never matches the raw black point. Black bars for example are always 0. And actual content also goes lower than minML. Same goes for maxML, maxCLL and the raw values can be higher than maxML. Best example for this is Mad Max Fury Road:
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 9918 cd/m2
Maximum Frame-Average Light Level : 3241 cd/m2
No idea why minML and maxML are even a part of HDR metadata. I talked to someone about it and they said that mastering luminance is meant for conent mastered on crappy displays. Though for professional content this is never the case. Most TVs base their rolloff or tone mapping on maxCLL and rarely on maxFALL never on anything else though. Monitors are a mixed bag though, some always tone map even without metadata in place, some behave like HGIG and don't process metadata at all. When the black point was still set to 0.005 nits from what I can tell native HDR content was first mapped to that and at one of the last steps mapped back down to 0. That change is barely visible though. And now you tell minML is taken into account 😨. I am bit confused as I thought it didn't or at least it doesn't look like it. I compared the raw video against what mpv/libplacebo does and I would say there is no difference even when the black point was still 0.005 nits. I guess that tone mapping step was/is invisible because the nits are so low?
This post was heavily edited because of an assumption I made. mp4box adds HDR metadata on the container level rather than bitstream level which mpv ignores. Stripping away the minML and maxML while leaving everything else untouched with a reencode "solves" this problem. Raw processing without influence of minML, maxML or other HDR metadata seems to be the way to go. Only VUI is important. I tried searching for other scenes affected by this but couldn't while I am able to replicate this issue though. I thought I somehow missed this while investigating #10558.
Stripping away the minML and maxML while leaving everything else untouched with a reencode "solves" this problem.
Yes but it's not a solution. HDR10 distributed media usually has that metadata present.
The simplest for now is probably to just not assume that the pixel coded values respect the min MDL. So by ignoring the source mastering display black point, or in the case of HDR10 passthrough, setting src min = dst min.
Detecting the real black floor is probably complicated.
some behave like HGIG and don't process metadata at all.
That is not how HGiG works. HGiG requires the source to not produce the values outside the display ability, so no tonemapping and gamut mapping is needed, even static one. But that is possible only in games, that render stuff on the fly. It uses Google SDK, even on Windows.
I can confirm that the raw black point for pretty much all HDR content I have watched always goes down to true black (0).
True black [body] is 64 in Y'. You can see it with ffmpeg -i file.mp4 uncompressedvideo.yuv and then opening it in YUView (select 420, 10 bit).
mp4box adds HDR metadata on the container level rather than bitstream level which mpv ignores.
That only works in VP9+webm. In all other cases metadata is in SEI inside HEVC bitstream.
I talked to someone about it and they said that mastering luminance is meant for content mastered on crappy displays.
Yes, but a) you are supposed to give the TV the ability to cut all the outside mastering black and white and outside gamut, which means you give unmodified values and metadata to HDMI b) there are many normal Blu-rays that have 0.0001 and BT.2020 mastering (like Star wars and The Matrix that has BT.2020). Some even have all 0 there, which means unknown, so that will preserve all blacks and whites.
I guess that tone mapping step was/is invisible because the nits are so low?
I think this was the case, yeah. I played around with the change for this issue on my own end and comparing 0.005 nits vs 0.000 nits is an extremely subtle change (*), smaller than the step from 0 to 1 in 8-bit SDR, and approaching the minimum just-noticeable difference. (You need dithering enabled to even see any difference!)
(*) In my typical fairly dimly-lit viewing environment.
https://www.microsoft.com/store/apps/9N7F2SM5D1LR for 22H2 is also only 0.005 nits, how did that happen?? Is there some limitation in windows 11?
My own end and comparing 0.005 nits vs 0.000 nits is an extremely subtle change
It is not. 66 vs 68 is subtle (65 being 0 nits black body and 68 being 0.0005 nits).
smaller than the step from 0 to 1 in 8-bit SDR, and approaching the minimum just-noticeable difference.
That is not how you measure JND, you need to be adapted. As in 10 minutes without light.
In my typical fairly dimly-lit viewing environment.
PQ reference conditions are not the same as SDR, PQ is almost dark room.