Anime4K_Clamp_Highlights.glsl causes darker colors when upscaling with gpu-next

[guidocella]

mpv Information

mpv v0.39.0-26-gc3d9243a3e Copyright © 2000-2024 mpv/MPlayer/mplayer2 projects
 built on Sep 28 2024 16:19:34
libplacebo version: v7.349.0 (v7.349.0-13-ga018ab04-dirty)
FFmpeg version: n7.0.2
FFmpeg library versions:
   libavcodec      61.3.100
   libavdevice     61.1.100
   libavfilter     10.1.100
   libavformat     61.1.100
   libavutil       59.8.100
   libswresample   5.1.100
   libswscale      8.1.100

Important Information

- Platform version: Latest Arch Linux
- GPU model, driver and version: can be reproduced on both AMD+Mesa and Nvidia proprietary drivers

Reproduction Steps

Download https://raw.githubusercontent.com/bloc97/Anime4K/7684e9586f8dcc738af08a1cdceb024cc184f426/glsl/Restore/Anime4K_Clamp_Highlights.glsl mpv --no-config --vo=gpu-next --glsl-shaders=Anime4K_Clamp_Highlights.glsl with any image or video Upscale the image or video

Expected Behavior

Colors stay the same

Actual Behavior

Colors become darker. In some images they even blend with the desktop wallpaper if the platform supports background transparency. This issue doesn't occur on vo_gpu or if upscaling with a shader.

Possibly related to https://github.com/mpv-player/mpv/issues/9524

Log File

mpv.log

Sample Files

Original screenshot:

Upscaled with no shader:

Upscaled with Anime4K_Clamp_Highlights.glsl:

I carefully read all instruction and confirm that I did the following:

• [X] I tested with the latest mpv version to validate that the issue is not already fixed.
• [X] I provided all required information including system and mpv version.
• [X] I produced the log file with the exact same set of files, parameters, and conditions used in "Reproduction Steps", with the addition of --log-file=output.txt.
• [X] I produced the log file while the behaviors described in "Actual Behavior" were actively observed.
• [X] I attached the full, untruncated log file.
• [X] I attached the backtrace in the case of a crash.

[guidocella]

--linear-downscaling=no prevents the change of colors.

The wallpaper blending (e.g. with mpv --glsl-shaders=Anime4K_Clamp_Highlights.glsl --vo=gpu-next --video-unscaled --background-color=0/0 https://0x0.st/XgNQ.jpg) is fixed by https://github.com/mpv-player/mpv/issues/9524#issuecomment-980260406, but even with that diff and disabling linear downscaling colors of images that would blend are much darker.

[kasper93]

This is probably effect of bt.1886 implementation in libplacebo

        const float lb = powf(csp_min, 1/2.4f);
        const float lw = powf(csp_max, 1/2.4f);
        const float a = powf(lw - lb, 2.4f);
        const float b = lb / (lw - lb);
        GLSL("color.rgb = "$" * pow(color.rgb + vec3("$"), vec3(2.4)); \n",
             SH_FLOAT(a), SH_FLOAT(b));

[   0.092][d][vo/gpu-next/libplacebo] [ 72] const float _8002 = float(0.0595848374068737); 
[   0.092][d][vo/gpu-next/libplacebo] [ 73] const float _8003 = float(0.8703105449676514); 
[   0.092][d][vo/gpu-next/libplacebo] [ 74] vec4 _8001(vec4 color) {
[   0.092][d][vo/gpu-next/libplacebo] [ 75] // pl_shader_linearize           
[   0.092][d][vo/gpu-next/libplacebo] [ 76] color.rgb = max(color.rgb, 0.0); 
[   0.092][d][vo/gpu-next/libplacebo] [ 77] color.rgb = _8003 * pow(color.rgb + vec3(_8002), vec3(2.4)); 
[   0.092][d][vo/gpu-next/libplacebo] [ 78] return color;
[   0.092][d][vo/gpu-next/libplacebo] [ 79] }

and

[   0.092][d][vo/gpu-next/libplacebo] [ 82] const float _4002 = float(0.0595848374068737); 
[   0.092][d][vo/gpu-next/libplacebo] [ 83] const float _4003 = float(1.149015188217163); 
[   0.092][d][vo/gpu-next/libplacebo] [ 84] vec4 _4001(vec4 color) {
[   0.092][d][vo/gpu-next/libplacebo] [ 85] // pl_shader_delinearize 
[   0.092][d][vo/gpu-next/libplacebo] [ 86] color.rgb = max(color.rgb, 0.0); 
[   0.092][d][vo/gpu-next/libplacebo] [ 87] color.rgb = pow(_4003 * color.rgb, vec3(1.0/2.4)) - vec3(_4002); 
[   0.092][d][vo/gpu-next/libplacebo] [ 88] return color;
[   0.092][d][vo/gpu-next/libplacebo] [ 89] }

[ghost]

--linear-downscaling=no prevents the change of colors.

linear-downscaling=no is a red herring I'm pretty sure, no? AFAIK, there's a behavioral quirk with libplacebo where this option controls all colorspace linearization, including the linearization that occurs before sigmoidal resampling. I don't remember if this was ever fixed or not. You just need to disable --sigmoid-upscaling to not crush the highlights.

Here's how the image looks like with --sigmoid-slope=11.5

[kasper93]

there's a behavioral quirk with libplacebo where this option controls all colorspace linearization, including the linearization that occurs before sigmoidal resampling. I don't remember if this was ever fixed or not.

No, it was not. linear-downscaling controls everything.

Here's how the image looks like with --sigmoid-slope=11.5

Yeah, but vo_gpu does the same. So not sure what's your point exactly?

Anyway, I was fixing something else, and made a fix for libplacebo https://code.videolan.org/videolan/libplacebo/-/merge_requests/676 might affect behavior here too, if you want to test, for me it doesn't affects this case.

linear-downscaling=no is a red herring I'm pretty sure, no?

Well, the difference in brightness is introduced by bt.1886 scaling in linearization, I don't believe it is sigmoid itself that makes different to vo_gpu.

[guidocella]

Anyway, I was fixing something else, and made a fix for libplacebo https://code.videolan.org/videolan/libplacebo/-/merge_requests/676 might affect behavior here too, if you want to test, for me it doesn't affects this case.

There is no difference with it.

[ghost]

So not sure what's your point exactly?

I dunno I thought the image looked cool. B) <--(clueless)

Well, the difference in brightness is introduced by bt.1886 scaling in linearization, I don't believe it is sigmoid itself that makes different to vo_gpu.

I think you're right. gpu and gpu-next look identical if I use --target-contrast=inf.

[mightyhuhn]

This is probably effect of bt.1886 implementation in libplacebo

        const float lb = powf(csp_min, 1/2.4f);
        const float lw = powf(csp_max, 1/2.4f);
        const float a = powf(lw - lb, 2.4f);
        const float b = lb / (lw - lb);
        GLSL("color.rgb = "$" * pow(color.rgb + vec3("$"), vec3(2.4)); \n",
             SH_FLOAT(a), SH_FLOAT(b));

[   0.092][d][vo/gpu-next/libplacebo] [ 72] const float _8002 = float(0.0595848374068737); 
[   0.092][d][vo/gpu-next/libplacebo] [ 73] const float _8003 = float(0.8703105449676514); 
[   0.092][d][vo/gpu-next/libplacebo] [ 74] vec4 _8001(vec4 color) {
[   0.092][d][vo/gpu-next/libplacebo] [ 75] // pl_shader_linearize           
[   0.092][d][vo/gpu-next/libplacebo] [ 76] color.rgb = max(color.rgb, 0.0); 
[   0.092][d][vo/gpu-next/libplacebo] [ 77] color.rgb = _8003 * pow(color.rgb + vec3(_8002), vec3(2.4)); 
[   0.092][d][vo/gpu-next/libplacebo] [ 78] return color;
[   0.092][d][vo/gpu-next/libplacebo] [ 79] }

and

[   0.092][d][vo/gpu-next/libplacebo] [ 82] const float _4002 = float(0.0595848374068737); 
[   0.092][d][vo/gpu-next/libplacebo] [ 83] const float _4003 = float(1.149015188217163); 
[   0.092][d][vo/gpu-next/libplacebo] [ 84] vec4 _4001(vec4 color) {
[   0.092][d][vo/gpu-next/libplacebo] [ 85] // pl_shader_delinearize 
[   0.092][d][vo/gpu-next/libplacebo] [ 86] color.rgb = max(color.rgb, 0.0); 
[   0.092][d][vo/gpu-next/libplacebo] [ 87] color.rgb = pow(_4003 * color.rgb, vec3(1.0/2.4)) - vec3(_4002); 
[   0.092][d][vo/gpu-next/libplacebo] [ 88] return color;
[   0.092][d][vo/gpu-next/libplacebo] [ 89] }

is the result accurate with just pow(source, 2.4) processing and pow(source, 1/2.4)?

isn't const float gamma_light = float(1.0/2.4) faster?

[llyyr]

The shader here is working as intended, or at least as written. It's a vo_gpu bug that it's not doing anything at all, whether the shader is present or not vo_gpu presents the exact same image.

The trick doesn't work, even though the condition stated in it is true in this case.

BT601/BT709 inverse Y′CbCr -> RGB matrix does have 1 for every row but Y itself has weighted contribution from each channel, so this shortcut doesn't really work. It only works if changing all three channels by the same amount produces the correct Y change.

I think to fix it you need to change each channel in proportion to their contribution to Y