h.264 playback with ffplay (v4l2_m2m) is choppy

[teknoman117]

Hello!

Thank you for your fantastic work on putting this together, I seriously appreciate your effort.

I pulled the image as of today (24/03/2020), and I'm having some trouble getting ffmpeg playback running at a real time framerate with a 1080p60 h.264 video (big buck bunny sample). The ffmpeg log was spamming "ff_v4l2_buffer_enqueue: VIDIOC_QBUF" errors (red text), and it was playing at about 20 fps, but very choppy. I tried on both an RPi 3 and RPi 4 and got similar results (low CPU usage, but choppy video). I used the vc4-fkms-v3d overlay on both with gpu_mem left at the default (128 MiB on the Pi 3, not sure about the Pi 4). I've included the ffplay output:

demouser@pi64 /run/media/demouser/Flash $ ffplay -vcodec h264_v4l2m2m -i bbb_sunflower_1080p_60fps_normal.mp4
ffplay version 4.2.2 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 9.2.0 (Gentoo 9.2.0-r3 p4)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-4.2.2/html --mandir=/usr/share/man --enable-shared --cc=aarch64-unknown-linux-gnu-gcc --cxx=aarch64-unknown-linux-gnu-g++ --ar=aarch64-unknown-linux-gnu-ar --optflags='-march=armv8-a+crc -mtune=cortex-a72 -ftree-vectorize -O2 -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --disable-optimizations --disable-libcelt --enable-nonfree --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --disable-gnutls --disable-gmp --enable-gpl --disable-hardcoded-tables --enable-iconv --disable-libtls --disable-libxml2 --disable-lzma --enable-network --disable-opencl --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --disable-vaapi --disable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --enable-libv4l2 --enable-libpulse --disable-libdrm --disable-libjack --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libcodec2 --disable-libdav1d --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libgme --disable-libgsm --disable-libaribb24 --enable-mmal --disable-libmodplug --disable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-libsrt --enable-librsvg --disable-ffnvcodec --enable-libvorbis --disable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --disable-cuda-llvm --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libtesseract --disable-lv2 --enable-libfreetype --disable-libvidstab --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --disable-libaom --disable-libopenh264 --disable-libsnappy --enable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --disable-libx265 --enable-libxvid --disable-gnutls --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-mmx --disable-mmxext --disable-sse --disable-sse2 --disable-sse3 --disable-ssse3 --disable-sse4 --disable-sse42 --disable-xop --cpu=armv8-a+crc --disable-doc --disable-htmlpages --enable-manpages
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    creation_time   : 2013-12-16T17:59:32.000000Z
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    composer        : Sacha Goedegebure
  Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn, 120 tbc (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Side data:
      audio service type: main
[h264_v4l2m2m @ 0x7f604dc520] driver 'bcm2835-codec' on card 'bcm2835-codec-isp'
[h264_v4l2m2m @ 0x7f604dc520] driver 'bcm2835-codec' on card 'bcm2835-codec-encode'
[h264_v4l2m2m @ 0x7f604dc520] driver 'bcm2835-codec' on card 'bcm2835-codec-decode'
[h264_v4l2m2m @ 0x7f604dc520] Using device /dev/video10
[h264_v4l2m2m @ 0x7f604dc520] driver 'bcm2835-codec' on card 'bcm2835-codec-decode'
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 2 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 3 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 4 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 5 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 6 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 7 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 8 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 9 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 10 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 11 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 12 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 13 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 14 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 15 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 16 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 17 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 18 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 19 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10f=0/0   
    Last message repeated 25 times
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
    Last message repeated 2 times
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 2 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 3 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 4 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 5 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 6 type 9 f=0/0   
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 7 type 9
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 0 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 1 type 10
ff_v4l2_buffer_enqueue: VIDIOC_QBUF : fd 20, index 8 type 9 f=0/0 

[sakaki-]

Hi @teknoman117,

ffplay does not have a very efficient pipeline, if using the default sdl output. Have you tried playing your target file with Applications→Multimedia→SMPlayer? You can force this to use either MMAL or v4l2_m2m endpoints for appropriate media types (click on the exclamation mark icon and browse through the video codecs).

[teknoman117]

I just tried SMPlayer. I forced the h264_v4l2m2m codec and I got a similar result (choppy video).

(I used the jellyfish sample I saw in your rpi forum post and the big buck bunny sample). However, SMTube works well for YouTube on the Pi4, but not on the Pi3

[sakaki-]

I'll take a look at the 1080p60 playback at some point later week.

[acroobat]

There are some libreelec patches that may help you. https://github.com/LibreELEC/LibreELEC.tv/tree/master/packages/multimedia/ffmpeg/patches

[sakaki-]

@acroobat -

thanks, merged most of these patches into the current ffmpeg (used on the forthcoming v1.6.0 image release); skipped hevc for now as had trouble building that on 64-bit.

best, sakaki

[sakaki-]

30 Oct 2020: sadly, due legal obligations arising from a recent change in my 'real world' job, I must announce I am standing down as maintainer of this project with immediate effect. For the meantime, I will leave the repo up (for historical interest, and since the images may be of use still in certain applications); however, there will be no further updates to the underlying binhost etc., nor will I be accepting / actioning further pull requests or bug reports from this point. Email requests for support will also have to be politely declined, so, please treat this as an effective EOL notice.

For further details, please see my post here.

Many thanks for your interest in this project!

With sincere apologies, sakaki ><