No hardware acceleration with cog/wpewebkit on imx6

[ghost]

No hardware acceleration with cog/wpewebkit on imx6. Not even getting 1fps for 720p video playback

[philn]

https://github.com/Igalia/meta-webkit/wiki/Providing-useful-GStreamer-Zero-copy-issue-reports

[vnd]

I guess I will piggyback this issue, since I'm experiencing similar problem: video played by cog is too slow.

Secondary problem is distorted colors (picture is reddish), which can be fixed by removing /usr/lib/gstreamer-1.0/libgstopengl.so. Another effect of removing that library is that VPU starts to actually do something, as seen by fired interrupts, however playback is still slow.

root@imx6dl-atlash:~# grep VPU /proc/interrupts
 25:          0          0       GPC   3 Edge      VPU_JPG_IRQ
 26:       1206          0       GPC  12 Level     VPU_CODEC_IRQ

Setup: i.MX6DL board, NXP stack (gpu-viv), gstreamer 1.16.3, plugins installed: base + good + bad + imx, Yocto mostly dunfell + master of meta-webkit. Video is full-screen and is matching screen resolution (480x800).

Video played via imxg2dvideosink is perfect (good speed, proper colors): gst-play-1.0 --videosink imxg2dvideosink video.mp4 All other options that I tried (imxeglvivsink, waylandsink, glimagesink) are either slow or non-functioning.

Attached are logs and dot files: cog.zip

[philn]

All other options that I tried (imxeglvivsink, waylandsink, glimagesink) are either slow or non-functioning.

Well that's part of the problem I suppose. WebKit's video sink is kinda similar to glimagesink so if that doesn't work well, it won't work much better in WebKit.

But anyways, looking at the pipeline dumps (thanks for providing those) I can see the sink used is not the GL one. So raw video caps are used, that means zero-copy rendering is disabled. You need to enable the USE_GSTREAMER_GL option in CMake. Is it enabled? Check the CMakeCache.txt file.

Also GStreamer 1.16 is old and un-maintained. You should switch to 1.20.x.

[philn]

Secondary problem is distorted colors (picture is reddish), which can be fixed by removing /usr/lib/gstreamer-1.0/libgstopengl.so.

That sounds either like a driver bug or some issue in GStreamer (though unlikely and even so, probably fixed since 1.16). I really recommend to use the latest stable GStreamer release, you won't get support for 1.16. Also the NXP folks are known for monkey-patching upstream GStreamer releases, which again won't be fixed by upstream GStreamer maintainers. Maybe you should "piggyback" this to whoever provides the imx video decoders. :)

[vnd]

From wpewebkit CMakeCache.txt: USE_GSTREAMER_GL:BOOL=ON

Not sure about 1.20.x gstreamer, since even master meta-freescale contains 1.18. Is it ok to use upstream gstreamer on i.MX6? I'm wondering why downstream version in meta-freescale exists.

At this point I'm more inclined to switch to opensource stack, since it looks like there are multiple (3 at least) reports of video playback being problematic with NXP stack.

[philn]

upstream should work, it's worth a try at least.

so you enabled gst-gl in wpewebkit at build time, but then you also earlier reported removing libgstopengl.so ... so effectively that disabled the webkit GL sink at runtime.

[vnd]

Switching to new gstreamer turned out to be problematic, as it effectively required switching entire yocto to kirkstone+ (or manually compiling gstreamear/adjusting gstreamer recipes/etc).

Instead I've switched to open-source stack (IMX_DEFAULT_BSP = "mainline") and everything worked out of the box.