vo_dmabuf_wayland: wayland VO displaying dmabuf buffers using vaapi or drm hwdec

[boxerab]

Wayland VO that can display images from either vaapi or drm hwdecs.

The PR adds the following changes:

1. a context_wayland Wayland context with no gl dependencies
2. no-op ra and dmabuf_interop objects - no-op because there is no need to map/unmap the drmprime buffer, and there is no need to manage any textures.

Tested on both x86_64 and rk3399 AArch64

cc @philipl

[CounterPillow]

Tested on an AMD Ryzen 7 Pro 3700U, appears to work (with usual no osc/osd caveats) on Weston aside from fullscreening refusing to cover Weston's launcher bar. KWin seemingly lacks NV12 support. Will test on aarch64 when I get the time.

[boxerab]

@Dudemanguy @philipl this PR is ready for a review, if you have time - would be much appreciated!

[Dudemanguy]

Looking at this in some more detail, I noticed that you added a video/out/context_wayland.c (you mentioned that in your opening post; missed it like a dummy). Would it be possible to rework it so all of it is handled directly in vo_dmabuf_wayland.c? I don't have a problem with the no-op ra and such, but this new context_wayland.c seems like an unnecessary level of abstraction to me. It's a bit confusing since context_platform.c code is generally meant to be under a specific API (opengl, vulkan, etc.).

[boxerab]

Looking at this in some more detail, I noticed that you added a video/out/context_wayland.c (you mentioned that in your opening post; missed it like a dummy). Would it be possible to rework it so all of it is handled directly in vo_dmabuf_wayland.c? I don't have a problem with the no-op ra and such, but this new context_wayland.c seems like an unnecessary level of abstraction to me. It's a bit confusing since context_platform.c code is generally meant to be under a specific API (opengl, vulkan, etc.).

You're right, contexts are designed to add support to a VO, in a generic fashion, for specific rendering APIs. But when running Wayland, the context also handles the compositor initialization/deinitialization. In this case, I just remove render API support, which is not needed, but keep the compositor init/deinit. This allows me to re-use the ra infrastructure to manage hwdecs, so that the vo can (almost) transparently support both vaapi and drm decoders.

My initial approach did not have a separate context, but then I had to make a number of changes to the existing ra and context code. Following advice from @philipl, I created a new ra and context and this was less invasive to existing code.

[Dudemanguy]

Hmm the problem with this is that --gpu-context=help lists wayland twice which is a bit confusing. It should at least have a different name. Not sure what exactly, maybe waylandbuf?

[boxerab]

Hmm the problem with this is that --gpu-context=help lists wayland twice which is a bit confusing. It should at least have a different name. Not sure what exactly, maybe waylandbuf?

const struct ra_ctx_fns ra_ctx_wayland_egl = {
    .type               = "opengl",
    .name               = "wayland",
    .reconfig           = wayland_egl_reconfig,
    .control            = wayland_egl_control,
    .wakeup             = wayland_egl_wakeup,
    .wait_events        = wayland_egl_wait_events,
    .update_render_opts = wayland_egl_update_render_opts,
    .init               = wayland_egl_init,
    .uninit             = wayland_egl_uninit,
};

this context should really be named waylandgl

[Dudemanguy]

Hmm, this VO errors out on my machine for some reason (--vo=dmabuf-wayland --hwdec=vaapi).

 (+) Video --vid=1 (*) 'Sora no Woto - Episode 1' (h264 1280x720 23.976fps)
File tags:
 Title: Sora no Woto - Episode 1
[hwupload] no support for this hw format
[hwupload] hardware format not supported
[autoconvert] can't find video conversion for yuv420p
Cannot convert decoder/filter output to any format supported by the output.
Could not initialize video chain.
Video: no video

Exiting... (Errors when loading file)

It works with vaapi-wayland though.

[philipl]

So, I finally got around to testing this by running weston. The main note is that all the work you've done in this PR to load the hwdec correctly means that everything works correctly even after my hwdec/filter changes all went in. In contrast to master where vaapi-wayland doesn't work properly.

On my Intel machine here, it recognises all the weird Intel packed formats and attempts to display them - all with visual problems, but recognisable all the same. Basic stuff like nv12 and p010 work just fine.

[t-8ch]

For me the console seems to be invisible. When pressing the console key it is not showing but when typing a command and pressing enter I get this in the log:

[input] Command 'foo' not found.
[input] Command was defined at console.

Edit: I don't know if this was also broken with vaapi-wayland because it doesn't seem to work for me.

[Dudemanguy]

That's expected. This doesn't have any OSD support.

[t-8ch]

Ah, indeed. Then sorry for the noise.

[focus64]

Pausing in the playback window (not in the terminal where mpv is launched) makes the playback window unresponsive to any inputs (mouse clicks, keyborads, etc).

I am running sway 1.7 on Arch Linux with Intel Tiger Lake graphics.

[boxerab]

Pausing in the playback window (not in the terminal where mpv is launched) makes the playback window unresponsive to any inputs (mouse clicks, keyborads, etc).

I am running sway 1.7 on Arch Linux with Intel Tiger Lake graphics.

Thanks for the report - this is now fixed if you rebase branch to latest.

[focus64]

Indeed! Thanks a lot for the quick fix.

A minor problem: resizing with mouse on Sway doesn't work well.

[boxerab]

A minor problem: resizing with mouse on Sway doesn't work well.

What exactly happens when resizing ?

[focus64]

Sorry I forgot to mention the resize was on a floating playback window.

What happens is really flaky, most of the time resize does not work, but sometimes it works but not exactly to the size I want.

[Dudemanguy]

@boxerab: Could you rebase this one against master? A couple of phillipl's PRs are in now that should be relevant here.

[Dudemanguy]

@boxerab: Thanks! The hwupload finally works for me now. I'm not sure if you missed the comment @philipl made earlier (github decided to autohide it even though the conversation isn't resolved) about the changes in f_hwtransfer.c. Basically, it was the part you commented out but I just got rid of it locally and it seems to just work™ for me on my machine. I have no way to test the drmprime part, but it works fine with vaapi.

Doing some testing right now and as far as I can tell this works as well as vaapi-wayland did for me. Resizing works fine for me (was noted earlier as possible issue).

[focus64]

I don't see resizing on a floating playback window gets improved. Perhaps I am the only unlucky one.

[Dudemanguy]

What issue are you seeing exactly? If it's just sway's borders being a bit goofy sometimes, that's normal and out of our control. But otherwise, it works just fine here. There's no reason why this should be any different (unless you're on nvidia or something I guess).

[focus64]

Here is my issue

I am on Intel Tiger Lake graphics.

I agree there should be no reason for me to be different, in theory. I am just reporting facts I see.

[focus64]

The previous vaapi-wayland vo had no problem with resizing on my system.

[Dudemanguy]

I saw that post, but I don't understand what that really means. If you have time and wouldn't mind recording a brief clip of what happens that would be appreciated.

[Dudemanguy]

Oh and also could you test weston and see if you have any problems there?

[focus64]

The way I do resize on a floating window in Sway is $mod+MouseRightClick. Hopefully you can reproduce my issue now.

[focus64]

I did test weston, it was the same.

[Dudemanguy]

The way I do resize on a floating window in Sway is $mod+MouseRightClick.

That's how I've been doing it the entire time. Works for me. ¯_(ツ)_/¯

[focus64]

Okay, can you please try a youtube video of 60 fps? Maybe my problem is more pronounced with 60 fps videos than 24 fps.

[Dudemanguy]

Okay so the resizing on a random 60 fps video does seem a bit buggy. As in, only during a few intervals while dragging does the resize actually occur and not smoothly through the whole process. I think it has to do with the toplevel_config not behaving as expected.

Seems normal if you pause the video and resize.

[Dudemanguy]

This fixes it for me.

diff --git a/video/out/vo_dmabuf_wayland.c b/video/out/vo_dmabuf_wayland.c
index 3ab77caabb..c961ebe3c8 100644
--- a/video/out/vo_dmabuf_wayland.c
+++ b/video/out/vo_dmabuf_wayland.c
@@ -196,13 +196,13 @@ static void flip_page(struct vo *vo)
 {
     struct vo_wayland_state *wl = vo->wl;
 
+    wl_surface_commit(wl->video_surface);
+    wl_surface_commit(wl->surface);
+
     if (!wl->opts->disable_vsync)
         vo_wayland_wait_frame(wl);
     if (wl->presentation)
        present_sync_swap(wl->present);
-
-    wl_surface_commit(wl->video_surface);
-    wl_surface_commit(wl->surface);
 }
 
 static void get_vsync(struct vo *vo, struct vo_vsync_info *info)
 }
 
 static void get_vsync(struct vo *vo, struct vo_vsync_info *info)

Edit: this actually better. Makes sense that the surface commits should be before vo_wayland_wait_frame.