raspberrypi
bcm2835-codec: H.264 video playback fails for 1080p
Describe the bug
I am trying to use the v4l2-mem2mem device to play AVC-encoded videos with hardware decoding on a Pi 3B or Zero 2W:
mpv --hwdec=v4l2m2m-copy video.mp4
# or
ffmpeg -codec:v h264_v4l2m2m -i video.mp4 -vcodec rawvideo -acodec copy -f matroska - | mpv -
Note: I was told in mpv's IRC channel that mpv also uses ffmpeg's libavcodec for the v4l2m2m decoder.
It works for some videos, but for others, playback never starts and the decoder seems to be stuck.
It seems to be related to the bitrate: I tried one video, downloaded from youtube in 720p and 1080p variants (see "Steps to reproduce"), and only the 1080p variant fails.
Steps to reproduce the behaviour
yt-dlp -f "ba[acodec^=mp4a]+bv[height=1080][vcodec^=avc1]" https://www.youtube.com/watch?v=K7dcSr04G8s
mpv --hwdec=v4l2m2m-copy Vinyl\ Cat\ \[K7dcSr04G8s].mp4
When switching to the 720p variant of the same video it works:
yt-dlp -f "ba[acodec^=mp4a]+bv[height=720][vcodec^=avc1]" https://www.youtube.com/watch?v=K7dcSr04G8s
Device (s)
Raspberry Pi 3 Mod. B, Raspberry Pi Zero 2 W
System
$ doas vcgencmd version
Dec 7 2024 13:02:17
Copyright (c) 2012 Broadcom
version 3858f977ab6d689a226ad24d26749266762b7160 (clean) (release) (start)
$ uname -a
Linux myhostname 6.12.13-0-rpi #1-Alpine SMP PREEMPT Thu Feb 13 22:15:47 UTC 2025 aarch64 Linux
Logs
I am attaching logs from mpv -v --hwdec=v4l2m2m-copy and from strace -y mpv --hwdec=v4l2m2m-copy (only the last couple of thousand lines).
mpv_v4l2m2m-copy_console.log
mpv_v4l2m2m-copy_strace.log
It always ends on this line until I press Ctrl+C and killall -9 mpv:
[..]
ppoll([{fd=17, events=POLLIN|POLLPRI|POLLOUT|POLLRDNORM|POLLWRNORM}], 1, NULL, NULL, 8
fd 17 refers to /dev/video10.
I took the strace logs last week; right now it ends like this (I noticed an extra munmap()):
futex(0x7fa4cc4404, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7fa5099f14, FUTEX_WAKE_PRIVATE, 1) = 1
clock_gettime(CLOCK_MONOTONIC_RAW, {tv_sec=79284, tv_nsec=624683192}) = 0
clock_gettime(CLOCK_MONOTONIC_RAW, {tv_sec=79284, tv_nsec=625218297}) = 0
ppoll([{fd=17, events=POLLIN|POLLPRI|POLLOUT|POLLRDNORM|POLLWRNORM}], 1, {tv_sec=0, tv_nsec=0}, NULL, 8) = 1 ([{fd=17, revents=POLLOUT|POLLWRNORM}], left {tv_sec=0, tv_nsec=0})
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9ee6c000
ppoll([{fd=17, events=POLLOUT|POLLWRNORM}], 1, {tv_sec=0, tv_nsec=0}, NULL, 8) = 1 ([{fd=17, revents=POLLOUT|POLLWRNORM}], left {tv_sec=0, tv_nsec=0})
ioctl(17, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, index=0, memory=V4L2_MEMORY_MMAP, m.offset=0xe63b6118, length=1, bytesused=0, flags=V4L2_BUF_FLAG_MAPPED|V4L2_BUF_FLAG_KEYFRAME|V4L2_BUF_FLAG_TIMESTAMP_COPY|V4L2_BUF_FLAG_TSTAMP_SRC_EOF, timestamp={tv_sec=3, tv_usec=366667}, ...}) = 0
ppoll([{fd=17, events=POLLOUT|POLLWRNORM}], 1, {tv_sec=0, tv_nsec=0}, NULL, 8) = 0 (Timeout)
ioctl(17, VIDIOC_QBUF, {type=V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, index=0, memory=V4L2_MEMORY_MMAP, m.offset=0xa4391368, length=1, bytesused=0, flags=V4L2_BUF_FLAG_MAPPED|V4L2_BUF_FLAG_QUEUED|V4L2_BUF_FLAG_KEYFRAME|V4L2_BUF_FLAG_TIMESTAMP_COPY|V4L2_BUF_FLAG_TSTAMP_SRC_EOF, ...}) = 0
munmap(0x7f9ee6c000, 4096) = 0
ppoll([{fd=17, events=POLLIN|POLLPRI|POLLOUT|POLLRDNORM|POLLWRNORM}], 1, NULL, NULL, 8
Additional context
The same RPi 3B with Arch Linux ARM has Kodi and ffmpeg with downstream patches for what I picked up as "the requests API". Here the same videos play flawlessly, but for my application I want to use Alpine without downstream patches.
Tested, and there is no issue on Raspberry Pi OS (at least with no windowing system hence rendering direct to DRM/KMS via mpv)
The request API is only relevant for the V4L2 stateless decoder API. H264 decode on Pi0-4 uses the V4L2 stateful decoder API
Mainline FFmpeg does the wrong thing with stateful decode in some situations, hence the decode stalls. That's not something that would get fixed in the kernel. (Gut feel is that you need https://github.com/jc-kynesim/rpi-ffmpeg/commit/7f8ee2091b5a3b804ace137180e61c95c304c779, but I get lost around FFmpeg patches).
Thank you for testing and providing some insight!
I see that the ffmpeg shipped by Raspberry Pi OS is also patched and I think it also contains the patch that you linked. (I looked into this archive and found /debian/patches/): https://archive.raspberrypi.org/debian/pool/main/f/ffmpeg/ffmpeg_5.1.6-0+deb12u1+rpt3.debian.tar.xz
So I tried applying https://github.com/jc-kynesim/rpi-ffmpeg/commit/b149dc92a2b23423e98f8412de27231b6987723b (same patch you linked to, but based on ffmpeg-6.1) to the APKBUILD in Alpine's aports. With two small fixes, it got built successfully.
It is now able to play the video in question, even though it drops frames, looks slow and goes out of sync with the audio. (Here is a log from mpv, not too interesting: mpv_v4l2m2m-copy_with_patched_ffmpeg.log)
But it seems to give some credibility to the position that the kernel driver shouldn't be blamed.