What USE_WPE_VIDEO_PLANE_DISPLAY_DMABUF really does?

[wolfallein]

Sorry if here is the wrong place to ask this.

I have a pinephone and would like to have hardware accelerated video playback with the WebKit engine.

Gstreamer already fully supports hardware decoding on Allwinner SOC's with the Cedrus driver. When using epiphany browser (or Minibrowser) I see hardware video decoding but video playback is still bad.

I believe the problem could be on the videosink part of the WebKit implementation that maybe is not using the hardware capabilities to show the video.

I couldn't find documentation for the switch USE_WPE_VIDEO_PLANE_DISPLAY_DMABUF and would like to know if this could help to fix the issue with video playback.

Thanks in advance

[philn]

What USE_WPE_VIDEO_PLANE_DISPLAY_DMABUF really does?

It's mostly an experiment initially meant to support Secure Video Path (DRM videos rendering). You shouldn't need this for hardware-accelerated video playback per se.

Can you read this page and report the informations needed to debug this? https://github.com/Igalia/meta-webkit/wiki/Providing-useful-GStreamer-Zero-copy-issue-reports

[wolfallein]

Thank you for the answer.

Here is some information:

GPU: Mali 400 MP2 Mesa: 21.3.5 with lima driver kernel: 5.15-sunxi64

Runing the command:

gst-launch-1.0 filesrc location=~/big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! v4l2slh264dec ! glimagesink

gives good results, and low CPU load as long as I have the environment variable GST_GL_API=gles2. If this is set to opengl (probably the default) I have low video performance although the CPU load is low.

To make sure GStreamer is not trying to use OpenGL API I compiled from the source with only gles2 support with the following meson options:

meson build -Dprefix=/usr -Dgst-plugins-base:gl_api=gles2 -Dgst-plugins-bad:v4l2codecs=enabled

mobian@mobian:~/src/gstreamer$ gst-inspect-1.0 --version
gst-inspect-1.0 version 1.19.3
GStreamer 1.19.3 (GIT)
Unknown package origin

After installing the compiled GStreamer I don't need to specify gles2 and video playback is always good when using gst-launch-1.0 or gst-play-1.0. It plays well even when I use autovideosink now, it probably uses glimagesink with gles2 in the end.

However, when using the browser (epiphany), it still gives poor performance with low CPU load, like when using the packaged GStreamer from Debian.

When running the browser, I also need to specify GDK_GL=gles otherwise I have:

** (epiphany:71955): WARNING **: 19:19:34.012: GDK is not able to create a GL context, falling back to glReadPixels (slow!): Unable to create a GL context

Now, getting the log files with MiniBrowser. For some reason when trying with epiphany I didn't get any log files in /tmp.

GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/tmp/gst.log GST_DEBUG_DUMP_DOT_DIR=/tmp /usr/lib/aarch64-linux-gnu/webkit2gtk-4.0/MiniBrowser http://samplemedia.linaro.org/H264/big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4

I'm trying to play mp4 directly to give the best chance. I get ~50% load in the CPU, and the video plays with some artifacts but plays.

Log files attached. I wonder if it possible to get closer to the performance of gst-play-1.0 or if this is really the best we could get with a browser.

Thank you!

log.tar.gz

[philn]

Can you post dot files for the gst-play case too please? I see the webkit video sink negotiates GL RGBA external-yes caps, I'm curious what glimagesink negotiates...

[wolfallein]

Hi, here it is. gst-play-log.tar.gz

Thank you

[philn]

Same caps in both cases... So next step would be to run perf on this and check what is the hotspot.

[wolfallein]

Let me know how I could run the tests. I don't have much experience using these tools.

Am I correct to say that gst-play and WebKit are using hardware decoder and the videosink seems to be the problem?

Do you think we should be getting more or less the same performance with webkit and gst-play? Or webkit will always be much slower?

Again, thanks a lot for the help. I know Pinephone is underpowered but I think that if the hardware capabilities are being used we should be able to get decent performance, especially because the CPU load is not at 100% when trying to play using browser.

Is it possible to compile webkit2gtk with only gles2 support? I'm afraid that it tries to use Opengl functions that it is not implemented in the hardware and ends up causing problems. Like gstreamer was doing if I don't specify GST_GL_API=gles2

[philn]

Let me know how I could run the tests. I don't have much experience using these tools.

Prefix the MiniBrowser command with perf record -g and then run perf report.

Am I correct to say that gst-play and WebKit are using hardware decoder and the videosink seems to be the problem?

Yes to the former. For the latter, I'm not sure yet...

Do you think we should be getting more or less the same performance with webkit and gst-play? Or webkit will always be much slower?

We should aim to have both on par.

Again, thanks a lot for the help. I know Pinephone is underpowered but I think that if the hardware capabilities are being used we should be able to get decent performance, especially because the CPU load is not at 100% when trying to play using browser.

Is it possible to compile webkit2gtk with only gles2 support? I'm afraid that it tries to use Opengl functions that it is not implemented in the hardware and ends up causing problems. Like gstreamer was doing if I don't specify GST_GL_API=gles2

I'm not familiar with that platform, but generally speaking I think we should keep default build options and let GStreamer/WebKit select the right GL/gles context at runtime.

[wolfallein]

Hi, attached you have the perf.data file together with new gst.log and *.dot files. log-gst-perf.tar.gz