Upgrade to FFmpeg 7.0

[mertalev]

Hi! I'm wondering what the current thoughts are on updating FFmpeg to 7.0? Following #306, the conclusion was to skip 6.1 and upgrade directly to 7.0. Is there anything blocking this upgrade?

For context, I ask because it has many improvements for Vulkan that I'd love to take advantage of (the long-term goal being to use Vulkan for decoding and filters across all devices). I can't use a normal FFmpeg build since my project relies on the RKMPP encoder here and possibly some of the other tweaks for hardware acceleration.

[gnattu]

ffmpeg 7.0 still has some severe regressions to be resolved and I personally want to wait for one or two more minor releases to have it fixed. We will also need to port all of our patches to that version which would take a lot of work as there are just too many breaking changes.

it has many improvements for Vulkan that I'd love to take advantage of (the long-term goal being to use Vulkan for decoding and filters across all devices)

This one is particularly fragile and very hard to maintain in the short term as Vulkan and its related filters like libplacebo has more than usual breaking changes from version to version. We are currently only enabling Vulkan filters for AMDGPU atm, and not using Vulkan for video decoding on any devices.

[mertalev]

Gotcha, sounds like it's too soon to use Vulkan so extensively. I actually made a branch changing everything (including cpu) to use a Vulkan pipeline before realizing it's so experimental haha. Now that you mention it, I was having some issues with scale_vulkan - which isn't documented anywhere - and there was a relevant fix just two weeks ago. The libplacebo docs also have a lot of settings that aren't in this FFmpeg build's libplacebo.

[nyanmisaka]

Hi @mertalev, jellyfin-ffmpeg is part of Jellyfin Server and we prioritize stability for the HWA platforms we have declared support for. Most of the features of jellyfin-ffmpeg6 had been tested for over a year before it was included in Jellyfin 10.9. When we feel ffmpeg7 is stable enough, we will provide jellyfin-ffmpeg7. If you are eager to move to ffmpeg7 immediately, you can try forking this and remove patches that you are not interested in to reduce the maintenance burden.

BTW full-featured Vulkan video support only exists in the latest hardware, even the most common Intel UHD graphics lack Linux driver support, and cross-platform support is terrible (e.g. Ryzen 5000 and UHD6xx are not supported on Windows). There is no reason to give up these hardware.

• Linux Intel: very limited feature, not usable for us
• LInux AMD: contributed by the community devs, not by AMD devs (encoding is only supported on VCN+)
• Windows Intel: only supports 11th Gen+ (UHD6xx not supported)
• Windows AMD: only supports RDNA1+ (Ryzen 5000G not supported)
• NVIDIA: Maxwell 2nd Gen+
• Apple: N/A
• Rockchip: N/A

With such fragmented support, I don't want to touch Vulkan video at all.

[mertalev]

Windows support isn't relevant for us, so it doesn't seem that bad for Linux.

That being said, the motivation of using Vulkan would be to streamline everything to a single pipeline. Based on your comments, I don't feel confident in putting all eggs in this basket with the current state of things.

[Andarwinux]

ffmpeg vulkan video is practically completely unusable, NVIDIA gets segmentation faults on all platforms, Windows Intel can't decode anything. Windows AMD performance is worse than dxva2.

[nyanmisaka]

ffmpeg vulkan video is practically completely unusable, NVIDIA gets segmentation faults on all platforms, Windows Intel can't decode anything. Windows AMD performance is worse than dxva2.

This is why we currently only use Vulkan compute on Linux AMD (RADV). Vulkan video still needs several years to stabilize.

[Ewillian9]

Currently trying ffmpeg 7 with 10.10, I see significant speed improvement in transcoding 1080p hevc 10b sdr into hevc (reduced bitrate) vs ffmpeg 6 with 10.9.11

From different sources, I went from 1100-1300 fps to 1700-2300fps. On Intel 12700 with UHD 770 quicksync

[nyanmisaka]

Currently trying ffmpeg 7 with 10.10, I see significant speed improvement in transcoding 1080p hevc 10b sdr into hevc (reduced bitrate) vs ffmpeg 6 with 10.9.11

From different sources, I went from 1100-1300 fps to 1700-2300fps. On Intel 12700 with UHD 770 quicksync

Usually you should check both ffmpeg logs to make sure they ran the same command line.

[Ewillian9]

Currently trying ffmpeg 7 with 10.10, I see significant speed improvement in transcoding 1080p hevc 10b sdr into hevc (reduced bitrate) vs ffmpeg 6 with 10.9.11

From different sources, I went from 1100-1300 fps to 1700-2300fps. On Intel 12700 with UHD 770 quicksync

Usually you should check both ffmpeg logs to make sure they ran the same command line.

I've checked, the only difference is ffmpeg 7 output the video roughly 100kbps higher than ffmpeg 6. Other than that everything is the same command wise. Same speed increase for every quality settings from 3mbps and below.

I only see this for hevc to hevc tho. Av1 to hevc, avc to hevc, av1 to avc, avc to avc and hevc to avc remain at similar speed.

EDIT: I've noticed another change, 10.9.11 + ffmpeg6, for a given source, output 240p video with 720kbps quality settings and 10.10 + ffmpeg7 for the same source and quality settings output 360p. Usually higher transcode output results in lower speed but not here, ffmpeg7 remains faster by a significant margin

[nyanmisaka]

Currently trying ffmpeg 7 with 10.10, I see significant speed improvement in transcoding 1080p hevc 10b sdr into hevc (reduced bitrate) vs ffmpeg 6 with 10.9.11 From different sources, I went from 1100-1300 fps to 1700-2300fps. On Intel 12700 with UHD 770 quicksync

Usually you should check both ffmpeg logs to make sure they ran the same command line.

I've checked, the only difference is ffmpeg 7 output the video roughly 100kbps higher than ffmpeg 6. Other than that everything is the same command wise. Same speed increase for every quality settings from 3mbps and below.

I only see this for hevc to hevc tho. Av1 to hevc, avc to hevc, av1 to avc, avc to avc and hevc to avc remain at similar speed.

EDIT: I've noticed another change, 10.9.11 + ffmpeg6, for a given source, output 240p video with 720kbps quality settings and 10.10 + ffmpeg7 for the same source and quality settings output 360p. Usually higher transcode output results in lower speed but not here, ffmpeg7 remains faster by a significant margin

I don't see such an improvement on Arc A380, can you share the ffmpeg logs?

[Ewillian9]

Currently trying ffmpeg 7 with 10.10, I see significant speed improvement in transcoding 1080p hevc 10b sdr into hevc (reduced bitrate) vs ffmpeg 6 with 10.9.11 From different sources, I went from 1100-1300 fps to 1700-2300fps. On Intel 12700 with UHD 770 quicksync

Usually you should check both ffmpeg logs to make sure they ran the same command line.

I've checked, the only difference is ffmpeg 7 output the video roughly 100kbps higher than ffmpeg 6. Other than that everything is the same command wise. Same speed increase for every quality settings from 3mbps and below.

I only see this for hevc to hevc tho. Av1 to hevc, avc to hevc, av1 to avc, avc to avc and hevc to avc remain at similar speed.

EDIT: I've noticed another change, 10.9.11 + ffmpeg6, for a given source, output 240p video with 720kbps quality settings and 10.10 + ffmpeg7 for the same source and quality settings output 360p. Usually higher transcode output results in lower speed but not here, ffmpeg7 remains faster by a significant margin

I don't see such an improvement on Arc A380, can you share the ffmpeg logs?

Sure, a bit later this day, I will

Edit: sorry I'm kinda in a hectic schedule rn, I'll come back when I can