vo=gpu-next does not use r16ui to map planes, so no yuv420p10 support (GLES)

[sfan5]

Important Information

• mpv v0.37.0-546-g474e213f52
• Android 14
• Mali-G68 w/ GLES 3.2

Reproduction steps

Play 10-bit video

Expected behavior

vo_gpu gpu.txt

Actual behavior

vo_gpu_next gpu2.txt here mpv does a conversion to gbrpf32 because it's the only >8-bit format the VO reports. this results in playback becoming a slideshow.

[haasn]

Seems like we need to allow for an extra UINT correction scaling factor, like what vo_gpu does in copy_image.

[kasper93]

To support this it is as simple as allowing the UINT formats, this is 5 minute patch. Though it needs special pass to support hooks, because they might not expect uint.

This works, for simple direct sampling case:

commit 387613bc823a9d4f8091b216ebfb694c351006ec
Author: Kacper Michajłow <[email protected]>
Date:   Mon Sep 16 21:58:26 2024 +0200

    utils/upload: allow integer upload for opengl

diff --git a/src/shaders/sampling.c b/src/shaders/sampling.c
index e7bddaf2..aff2c2e0 100644
--- a/src/shaders/sampling.c
+++ b/src/shaders/sampling.c
@@ -108,6 +108,9 @@ static bool setup_src(pl_shader sh, const struct pl_sample_src *src,
     if (scale)
         *scale = PL_DEF(src->scale, 1.0);
 
+    if (src->tex->params.format->type == PL_FMT_UINT)
+        *scale *= 1.0 / ((1ull << 16) - 1);
+
     if (comp_mask) {
         uint8_t tex_mask = 0x0Fu;
         if (src->tex) {
@@ -277,8 +280,8 @@ bool pl_shader_sample_direct(pl_shader sh, const struct pl_sample_src *src)
     if (!setup_src(sh, src, &tex, &pos, NULL, NULL, NULL, NULL, &scale, true, BEST))
         return false;
 
-    GLSL("// pl_shader_sample_direct                            \n"
-         "vec4 color = vec4("$") * textureLod("$", "$", 0.0);   \n",
+    GLSL("// pl_shader_sample_direct                                  \n"
+         "vec4 color = vec4("$") * vec4(textureLod("$", "$", 0.0));   \n",
          SH_FLOAT(scale), tex, pos);
     return true;
 }
diff --git a/src/utils/upload.c b/src/utils/upload.c
index 4a94fd18..2c0ca185 100644
--- a/src/utils/upload.c
+++ b/src/utils/upload.c
@@ -187,7 +187,9 @@ pl_fmt pl_plane_find_fmt(pl_gpu gpu, int out_map[4], const struct pl_plane_data
         pl_fmt fmt = gpu->formats[n];
         if (fmt->opaque || fmt->num_components < num)
             continue;
-        if (fmt->type != data->type || fmt->texel_size != data->pixel_stride)
+        if (fmt->texel_size != data->pixel_stride)
+            continue;
+        if (fmt->type != data->type && fmt->type != PL_FMT_UINT)
             continue;
         if (!(fmt->caps & PL_FMT_CAP_SAMPLEABLE))
             continue;

[peerless2012]

The patch above works, gpu-next + gl + 10bit + soft decode. @haasn Is there any plan to fix this issue? It's very helpful for me, ffmpeg on my platform not support hardware decode.

[peerless2012]

@kasper93 and @haasn When will the patch above will merge into the main repo? It's very useful.

[haasn]

@kasper93 can you submit it as a MR?

Also maybe change the (1 << 16) to the bit depth of the format.

[kasper93]

@haasn This needs another (sub)pass to convert uint to unorm. The above patch is only for direct sampling case, it likely would break if you use different scaling or custom hooks.

[haasn]

@kasper93

diff --git a/src/renderer.c b/src/renderer.c
index e1f42f30..e05aca4c 100644
--- a/src/renderer.c
+++ b/src/renderer.c
@@ -1659,7 +1659,7 @@ static bool pass_read_image(struct pass_state *pass)
             did_merge = true;
         }
 
-        if (!did_merge)
+        if (!(did_merge || sti->fmt->type == PL_FMT_UINT))
             continue;
 
         if (!img_tex(pass, &sti->img)) {

This may help, although I think there still might be cases not caught by this logic. I think it will be sufficient if we also add a check to want_merge when scaling is requested but the input texture does not support linear interpolation and is uint. And we probably also want to add the correct scale factor to the main scaler as well.

[kasper93]

I pushed https://code.videolan.org/videolan/libplacebo/-/merge_requests/721

I didn't force uint formats, but moved them up in the list, hopefully they will be selected. Not tested. Feel free to test, I don't have env to test it on device.

[sfan5]

You don't need Android, just GLES should suffice. I can also test it if you want.

[kasper93]

Conversion should work fine, format selection likely will not. Feel free to suggest changes that works for your device.

[sfan5]

indeed still selects gbrpf32: https://0x0.st/8npa.txt

I don't think there's anything device specific to this. If vo_gpu_next properly offers a better format than 32-bit RGB float, mpv will prefer it.

[kasper93]

indeed still selects gbrpf32: https://0x0.st/8npa.txt

I don't think there's anything device specific to this. If vo_gpu_next properly offers a better format than 32-bit RGB float, mpv will prefer it.

My hardware doesn't expose rgba32f, so I cannot really test this without a "device". Does this fix UINT selection for you?

diff --git a/video/out/vo_gpu_next.c b/video/out/vo_gpu_next.c
index 6b6467ac6e..89a6e20e91 100644
--- a/video/out/vo_gpu_next.c
+++ b/video/out/vo_gpu_next.c
@@ -502,9 +502,11 @@ static int plane_data_from_imgfmt(struct pl_plane_data out_data[4],
         }

         data->pixel_stride = desc.bpp[p] / 8;
-        data->type = (desc.flags & MP_IMGFLAG_TYPE_FLOAT)
-                            ? PL_FMT_FLOAT
-                            : PL_FMT_UNORM;
+        data->type = PL_FMT_UNORM;
+        if (desc.flags & MP_IMGFLAG_TYPE_FLOAT)
+            data->type = PL_FMT_FLOAT;
+        if (desc.flags & MP_IMGFLAG_TYPE_UINT)
+            data->type = PL_FMT_UINT;
     }

     if (any_padded && !out_bits)

EDIT: Also please check the log if it complains about linear sampling.

[kasper93]

The issue has been fixed upstream, @sfan5 could you confirm it's working for you with the above patch, so we can merge the fix on mpv side too?

[sfan5]

@kasper93 works: https://0x0.st/8n5M.txt does "Failed dispatching scaler" mean it won't support any scaler and fall back to bilinear?

[sfan5]

Fixed by https://code.videolan.org/videolan/libplacebo/-/merge_requests/721.