KODI fails to play AV1 media encoded with "keyframe-filtering=2"

[Hlsgs]

Bug report

Describe the bug

Here is a clear and concise description of what the problem is:

KODI plays only the audio from a file if the video is AV1 and encoded with --enable-keyframe-filtering=2. This is reproducible across platforms and irrespective of whether HW acceleration is enabled or not. I have tested 20.1 release on Windows 10 and Android 12 as well as the latest nightly on Windows 10. All test were done both with HW acceleration on and off.

Here are a couple of sample files to test from an FFMPEG Bug that may be related and is supposed to be solved:

• keyframe-filtering=1 encoded file
• keyframe-filtering=2 encoded file

Note1: Above files do not contain audio, but KODI will still fail to play the keyframe-filtering=2 one, while having no issue with the keyframe-filtering=1 one. Note2: Despite the above mentioned fix, the latest FFMPEG still fails to remux such files, silently losing the video in the process. Which is interesting because that seems to be exactly what KODI is "losing" as well.

Expected Behavior

Here is a clear and concise description of what was expected to happen:

AV1 playback should work irrespective of the encoder settings used to encode the file.

Actual Behavior

Any AV1 file encoded with keyframe-filtering=2 will fail, and this seems to be platform agnostic.

Possible Fix

Some FFMPEG based players seem to have this fixed(VLC), while other do not(MPC-BE). Perhaps a fix can be found there or perhaps FFMPEG will fix this upstream. Seems to me like this is an FFMPEG bug, but I could be dead wrong, of course :)

To Reproduce

Steps to reproduce the behavior:

1. Play any AV1 file encoded with keyframe-filtering=2 or this sample from FFMPEG
2. The bug will be readily apparent on any platform/config and irrespective of HW acceleration being used or not, I believe.

Debuglog

The debuglog can be found here:

Not needed, as this is readily reproducible for anyone.

Screenshots

Here are some links or screenshots to help explain the problem:

Additional context or screenshots (if appropriate)

Here is some additional context or explanation that might help:

Your Environment

Used Operating system:

• [x] Android

• [ ] iOS

• [ ] tvOS

• [ ] Linux

• [ ] macOS

• [x] Windows

• [ ] Windows UWP

• Operating system version/name:

• Kodi version: Android 1(v20.1), Win 10 x64(v20.1 and 20230520-2f793e7f-master nightly).

note: Once the issue is made we require you to update it with new information or Kodi versions should that be required. Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.

[cogman]

Can confirm, I have the same issue with kodi 20.2 on debian with dav1d 1.2.

This appears to be fixed in ffmpeg 5.1+ with this commit

https://github.com/FFmpeg/FFmpeg/commit/3e186148ca9ac0c47cec253fdea62b48c9feadd2

[Hlsgs]

Can confirm, I have the same issue with kodi 20.2 on debian with dav1d 1.2.

This definitely seems to be platfom/Kodi version agnostic.

This appears to be fixed in ffmpeg 5.1+ with this commit

FFmpeg/FFmpeg@3e18614

I wouldn't be so sure about that, as that seems to be the same fix I've linked to. To quote from my own post, in which I'm referring to ffmpeg 6.0.0:

Note2: Despite the above mentioned fix, the latest FFMPEG still fails to remux such files, silently losing the video in the process. Which is interesting because that seems to be exactly what KODI is "losing" as well.

Anyway, hope this gets more traction so it'll get triaged sooner rather than later.

[Hlsgs]

Tested the latest nightly(KodiSetup-20230930-72dfef5b-master-x64.exe) and the issue is still present. If there have been updates to FFMPEG/dav1d, they do not seem to include a fix for this.

[Rainbowlemon]

Have just encountered this same issue trying to play AV1s - completely freezing video but audio plays in the background. I'm on linux with jellyfin addon for kodi.

[neo1973]

The issue still occurs after Kodi switched to FFmpeg 6.0.1. Even when building Kodi with FFmpeg 6.1 the sample file can't be played. mpv and ffplay on the other hand can play the file. I thinks it's save to say the issue is with Kodi and not FFmpeg.

[Hlsgs]

The issue still occurs after Kodi switched to FFmpeg 6.0.1. Even when building Kodi with FFmpeg 6.1 the sample file can't be played. mpv and ffplay on the other hand can play the file. I thinks it's save to say the issue is with Kodi and not FFmpeg.

I just tried remuxing the keyframe-filtering=2 encoded file with ffmpeg version 6.1-essentials_build and it still fails silently to include the video in the output file. This can, of course, be unrelated and I'm grasping at straws, but, the "similarity" with KODI's bahavior seems interesting to me and suggests that either the issue is with ffmpeg's decoder and the mentioned players have it fixed on their own or the issue is with ffmpeg's splitter, which said players are not using or have, again, fixed on their own. To be honest, I don't even know if KODI uses ffmpeg's splitter, so please take my "analysis" with a huge grain of salt :)

[rsalvaterra]

Has there been any progress here? I have several AV1-encoded files which are unplayble in Kodi (either in hardware or software), for some unfathomable reason, but which mpv plays just fine (either in hardware or software). However, I don't know if that's related to this particular issue, since I don't know what the encoding parameters were.

EDIT: I forgot to mention that this is in LibreELEC 12 daily (GBM), thus Linux.

[Hlsgs]

Has there been any progress here? I have several AV1-encoded files which are unplayble in Kodi (either in hardware or software), for some unfathomable reason, but which mpv plays just fine (either in hardware or software). However, I don't know if that's related to this particular issue, since I don't know what the encoding parameters were.

EDIT: I forgot to mention that this is in LibreELEC 12 daily (GBM), thus Linux.

No, there hasn't. Just tested the latest v21 Nightly(KodiSetup-20240305-6a0aaf8c-master-x64.exe) and it exibits the same behavior. If you're getting audio but no video on those AV1 files(and vlc/mpv plays them fine), it most definitely is this issue.

I was hoping this would get on the radar of a team member in time for a fix in v21, but this seems less and less likely, as the next step seems to be a RC.

[rsalvaterra]

Is it a reproducibility issue? The videos I mentioned in my previous post are copyrighted, so I obviously can't just point people to them, and I'm not really qualified to judge if this (debugging Kodi) is a fair use situation (IANAL). @neo1973?

[Hlsgs]

It's 100% reproductible on any platform Kodi runs on. Also, no need whatsoever to share your files, anyone can just test with the samples I've provided in my OP. The problem that this report just isn't getting on anyone relevant's radar.

[Chronosaurus42]

Unfortunately, the problem still exists in version 21.0

[karloff10]

Until this problem is fixed in kodi we can use a workaround with External players this should work: $HOME/.kodi/userdata/playercorefactory.xml

<playercorefactory>
  <players>
    <player name="MPV" type="ExternalPlayer" audio="false" video="true">
      <filename>/usr/bin/mpv</filename>
      <args>--fs=yes "{1}"</args>
      <hidexbmc>false</hidexbmc>
    </player>
  </players>
  <rules action="prepend">
    <rule video="true" filename=".*AV1.*" player="MPV"/>
  </rules>
</playercorefactory>

Unfortunately, I was not able to use something like videocodec="AV1" inside the rule, so all files had to be a AV1 inside the filename to get played with mpv.

[Hlsgs]

@Chronosaurus42 Yes, and it'll probably stay that way until we find a way to raise awerness about it.

@karloff10 Nice, buit I don't think one can do such a thing on Android. Regardless, this is probably something that's trivial to fix for the right people, so rather than "Until this problem is fixed in kodi" we should say "Until this gets on the right people's radar". I guess there's just not enough interest in AV1 as of now :|

[zelmon64]

I've run into this issue as well but I've found that the keyframe-filtering=2 test file works on Android when using hardware acceleration. I'm using a Samsung Galaxy Note 9 with an Exynos chipset (SM-N960F) running Android 13.0.0 and Kodi 20.3.0.

[Hlsgs]

@zelmon64 That's the first instance and platform where I've heard of it working. Are you sure you tested right? Anyway, thanks for the input. It's probably useful info for whomever will take a dive into this.

[zelmon64]

Are you sure you tested right?

I'm pretty sure. When I have either of the hardware acceleration options enabled (MediaCodec (Surface) or MediaCodec) it plays with the video decoder reported as amc-av1(S)(HW) or amc-av1(HW). When they are both disabled it fails to play.

[Hlsgs]

Are you sure you tested right?

I'm pretty sure. When I have either of the hardware acceleration options enabled (MediaCodec (Surface) or MediaCodec) it plays with the video decoder reported as amc-av1(S)(HW) or amc-av1(HW). When they are both disabled it fails to play.

Hmm. There's probably something different in that chipset's video pipeline that bypasses the part of Ffmpeg/KODI that causes this. Interesting. And probably more interesting for whomever actually understands this stuff better :)

[a1rwulf]

We are a (very) small team of active devs, I made some initial effort to implement AV1 hardware accelaration but lack the time to contribute anything at the moment - I just want to let you know, that we see your comments, but there's just nobody available to work on it atm.

[Hlsgs]

We are a (very) small team of active devs, I made some initial effort to implement AV1 hardware accelaration but lack the time to contribute anything at the moment - I just want to let you know, that we see your comments, but there's just nobody available to work on it atm.

Thank you for taking the time to let me know! Much appreciated.

One "quick" question: am I to assume from you answer that this is a software decode issue and HW accelerated android devices should not have this problem? I was testing SW decode on android(don't own a hw decode capable device) and was presuming that I'm testing HW decode on Windows, but your answer suggests that's not the case. That being said, there are users above confirming this on Linux, which should have HW decode if they have the GPU, so there's that.

[a1rwulf]

That might have been confusing, sorry. I have no idea which device, driver and acceleration combination would work with the files in question.

I just wanted to emphasize that the only efforts being done in AV1 since 2022 were about getting hw acceleration into platforms like Linux, Android, Windows as well as the InputStream addon and enabling DolbyVision profiles for AV1 encoded files. I might be wrong, but that's what a 5min search for AV1 in our github PR's says.

[Hlsgs]

That might have been confusing, sorry. I have no idea which device, driver and acceleration combination would work with the files in question.

I just wanted to emphasize that the only efforts being done in AV1 since 2022 were about getting hw acceleration into platforms like Linux, Android, Windows as well as the InputStream addon and enabling DolbyVision profiles for AV1 encoded files. I might be wrong, but that's what a 5min search for AV1 in our github PR's says.

Alright, thanks for the explainer. I guess I'll have to wait and see when the next Chromecast comes out eventually. I'll definitely come back and post here when I get my hands on an AV1 capable device, be that said Chromecast or otherwise.