Add rkmpp support

[tandy-1000]

TODO:

• [x] Add rkmpp for arm64 build
• [x] Enable the build flag

Changes

• For arm64 platform the rkmpp dependency is added

Issues Fixes https://github.com/jellyfin/jellyfin-ffmpeg/issues/34

[nyanmisaka]

#161 has been merged, you can rebase to avoid the CI error.

[Shadowghost]

Can we rebase and squash before merging please :)

[tandy-1000]

Squashed!

[nyanmisaka]

Once the CI is done, you need to verify that artifact works on a rockchip SBC.

[tandy-1000]

I've installed the artifact, how do I test it?

[nyanmisaka]

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/rkmppdec.c

/usr/lib/jellyfin-ffmpeg/ffmpeg -decoders | grep rkmpp

hwdec: /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264_rkmpp -i h264.mp4 -an -sn -f null - /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v hevc_rkmpp -i hevc.mp4 -an -sn -f null -

swdec: /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264 -i h264.mp4 -an -sn -f null - /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v hevc -i hevc.mp4 -an -sn -f null -

[tandy-1000]

(WIP)

Decoders

$ /usr/lib/jellyfin-ffmpeg/ffmpeg -decoders | grep rkmpp
ffmpeg version 5.0.1-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --enable-rkmpp --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
 V..... h264_rkmpp           h264 (rkmpp) (codec h264)
 V..... hevc_rkmpp           hevc (rkmpp) (codec hevc)
 V..... vp8_rkmpp            vp8 (rkmpp) (codec vp8)
 V..... vp9_rkmpp            vp9 (rkmpp) (codec vp9)

H264 hw decode

$  /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264_rkmpp -i Big_Buck_Bunny_1080_10s_1MB.mp4 -an -sn -f null -
ffmpeg version 5.0.1-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --enable-rkmpp --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Big_Buck_Bunny_1080_10s_1MB.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
  Duration: 00:00:10.00, start: 0.000000, bitrate: 837 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 832 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize MPP context (code = -1).
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize RKMPP decoder.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> wrapped_avframe (native))
Error while opening decoder for input stream #0:0 : Unknown error occurred

[nyanmisaka]

[h264_rkmpp @ 0xaaab04df3320] Failed to initialize MPP context (code = -1).
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize RKMPP decoder.

Add -v debug to that cli. Also you have to make sure the rockchip model is supported by rkmpp.

https://github.com/rockchip-linux/mpp/blob/786b79f9767d24bed618be60046546b508f9190e/inc/mpp_err.h#L27 https://github.com/rockchip-linux/mpp/blob/17a47899be092eb019729dcf28b3030485d35c75/mpp/mpi.cpp#L432

Actually I'm not familiar with rkmpp. You have to debug on your own.

[Quedale]

I doubt mpp support can be added straight out of jellyfin's setup. It requires a specific rockchip linux kernel 4.4.x.

Fortunately armbian maintains a legacy build for that kernel which makes the initial setup easy, but without this very specific kernel, the mpp library will fail to initialize. (Mainline linux doesn't have /dev/vpu_service available)

Assuming that full rkmpp integration is just a holygrail, would there be a way to force jellyfin to "try" to use ffmpeg's mpp codec?

So far, I got a custom ffmpeg build with mpp enabled. I also changed jellyfin's default ffmpeg binary, but I can't get hw accel to work with jellyfin. Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

[tandy-1000]

Assuming that full rkmpp integration is just a holygrail, would there be a way to force jellyfin to "try" to use ffmpeg's mpp codec?

So far, I got a custom ffmpeg build with mpp enabled. I also changed jellyfin's default ffmpeg binary, but I can't get hw accel to work with jellyfin. Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

I'm pretty sure that's what this PR is trying to do, I guess I got stalled because I'm not using that specific kernel.

[nyanmisaka]

Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

This PR has just enabled rkmpp encoder in our custom ffmpeg.

To utilize rkmpp you still need to modify the source code of Jellyfin to add support for it. Or use a ffmpeg wrapper to replace the software encoder libx264 with rkmpp encoder.

[Quedale]

Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

This PR has just enabled rkmpp encoder in our custom ffmpeg.

To utilize rkmpp you still need to modify the source code of Jellyfin to add support for it. Or use a ffmpeg wrapper to replace the software encoder libx264 with rkmpp encoder.

Thanks, I wasn't sure if I was missing anything obvious.

I suppose the solution doesn't necessarily need to be specific for rkmpp. Adding the ability to define ffmpeg's encoder/decoder wouldn't be a terrible idea.

I'm thinking that if we can define decoder/encoder separately, it could leverage 2 separate hardware.

The reason I'm thinking like this is because on RK3399, hevc and h264 shares the same VPU. Which limits its transcoding capability to 1080p@30fps if i'm not mistaken. (I read that somewhere, sorry I don't have the source) The RockPro64 for instance has a PCI-E port, which could be used to split that load.

I might be thinking too far ahead. I just thought I'd share the thought.

Jumps down the rabbit hole

[dmarkey]

Hey, is this going anywhere, or am I able to test it out somehow?

[IngwiePhoenix]

I have a FriendlyElec NanoPi R6s.

Is there any way I can help with testing? Hardware accellerated transcoding would be really neat! :)

[nyanmisaka]

I have a FriendlyElec NanoPi R6s.

Is there any way I can help with testing? Hardware accellerated transcoding would be really neat! :)

https://github.com/jellyfin/jellyfin-ffmpeg/issues/34#issuecomment-1522612368

[nyanmisaka]

Superseded by #318