gphoto2 through ffmpeg with VA-API hardware acceleration produces a broken result

[archerallstars]

Describe the bug

I tried to use gphoto2 with VA-API hardware acceleration with:

gphoto2 --stdout --capture-movie | ffmpeg -hwaccel vaapi -c:v mjpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 2 -f v4l2 /dev/video14

The result streaming to my vl42loopback device, /dev/video14, is broken whether I specified the -c:v mjpeg, as shown in my screenshot below:

The acceleration works on my iGPU (UHD 620 - KBL GT2) in i5-8250u, though.

I tested to check whether the ffmpeg (5.1-2.7) in my system (openSUSE Tumbleweed) is broken. I can convert my mp4 file to a mov file without any issue using:

ffmpeg -hwaccel vaapi -i input.mp4 -c:v mjpeg output.mov

It seems, ffmpeg is working right, the converted video file is not broken, and hardware acceleration works fine also.

If I am not using hardware acceleration, gphoto2 works fine. But the CPU usage would be a lot higher, as shown in the screenshot below:

Name the camera This is not related to the camera.

libgphoto2 and gphoto2 version output of: gphoto2 --version

gphoto2 2.5.28

Copyright (c) 2000-2021 Marcus Meissner and others

gphoto2 comes with NO WARRANTY, to the extent permitted by law. You may
redistribute copies of gphoto2 under the terms of the GNU General Public
License. For more information about these matters, see the files named COPYING.

This version of gphoto2 is using the following software versions and options:
gphoto2         2.5.28         gcc, popt(m), exif, cdk, no aa, jpeg, readline
libgphoto2      2.5.30         standard camlibs (SKIPPING lumix), gcc, no ltdl, EXIF
libgphoto2_port 0.12.1         iolibs: disk ptpip serial usb1 usbdiskdirect usbscsi, gcc, no ltdl, EXIF, USB, serial lockdev locking

To Reproduce

1. Connect the camera, make sure to unmount the camera in the system.
2. Launch gphoto2 with VA-API hardware acceleration by: gphoto2 --stdout --capture-movie | ffmpeg -hwaccel vaapi -c:v mjpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 2 -f v4l2 /dev/video14, but at your v4l2loopback device. Mine is /dev/video14. You can check this by v4l2-ctl --list-devices.
3. Open VLC > Media > Open Capture Device to test gphoto2 stream.

[archerallstars]

As a workaround for this issue, I found that I can use QSV instead of VA-API for hardware acceleration. The command is:

gphoto2 --stdout --capture-movie | ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -c:v mjpeg_qsv -i - -vcodec rawvideo -pix_fmt yuv420p -threads 2 -f v4l2 /dev/video14

My v4l2loopback device is /dev/video14. You can check yours with v4l2-ctl --list-devices.

However, this workaround will not work with AMD guys. Therefore, hardware acceleration with VA-API would be more preferable.

[archerallstars]

As another workaround for anyone having this issue and trying to make hardware acceleration work with FFmpeg. Forget about hardware acceleration with FFmpeg. You can use GStreamer instead by:

gphoto2 --stdout --capture-movie | gst-launch-1.0 fdsrc ! decodebin3 name=dec ! queue ! videoconvert ! v4l2sink device=/dev/video14

I am not able to make VA-API work with GStreamer yet. However, the performance and resource usage with an unaccelerated GStreamer is far better than an accelerated FFmpeg. There's zero latency on my weak iGPU (UHD 620). I recommend everyone to use gPhoto2 through the GStreamer pipeline instead.