Android MediaCodec Decoder Does Not Work

[jaruba]

Description

When trying to decode with the mediacodec hwaccel decoder, the decoder gives a lot of errors, it seems to only decode the first frame or first few video frames after which it fails at all the rest.

Expected behavior

The mediacodec decoder should decode all video frames.

Current behavior

The mediacodec decoder gives many errors. When starting playback of the resulted video, sometimes it looks like it decoded the first few frames only, other times it looks like it decoded only the first frame.

To Reproduce

Attempt using any of the following ffmpeg args:

-c:v hevc_mediacodec -i https://dl5.webmfiles.org/video-h265.mkv -avoid_negative_ts disabled -c:v libx264 -c:a copy -movflags frag_keyframe+empty_moov -f mp4

-hwaccel mediacodec -c:v hevc_mediacodec -i https://dl5.webmfiles.org/video-h265.mkv -avoid_negative_ts disabled -c:v libx264 -c:a copy -movflags frag_keyframe+empty_moov -f mp4

Screenshots

Not applicable.

Logs

https://pst.klgrth.io/paste/puqyk

Environment

• Platform: Android
• Architecture: arm-v7a
• Version: ffmpeg-kit-full-gpl:4.5.LTS, ffmpeg-kit-full-gpl:4.5.1.LTS, ffmpeg-kit-full-gpl:5.1.LTS
• Source branch: -
• Xcode version: -
• Cocoapods version: -
• Android Studio version: -
• Android NDK version: -

Other

I found this exact error Error while decoding stream #0:0: Try again in only one other place (google gives 1 result), and that is at another issue created at mobile-ffmpeg (older name for ffmpeg-kit mobile builds), an issue created by @Vigrond which had no resolution: https://github.com/tanersener/mobile-ffmpeg/issues/707

I have attempted decoding with 3 different versions of ffmpeg-kit: v4.5.LTS, v4.5.1.LTS, v5.1.LTS and they all result in the same outcome.

I have tried with many HEVC videos and they all result in the same outcome also.

Tests have been done on an actual device (not emulators), the testing device is a Pixel 5 running Android 13 (latest Android version with all system updates to date) - Build Number: TP1A221105.002

At first I presumed that this could be a deadlock, so I tried tweaking the input args with -threads, but this did not seem to make any difference.

Thanks in advance!

[tanersener]

I'm not sure what we can do here in ffmpeg-kit about this. The source code of MediaCodec decoder comes from ffmpeg. And, we are not modifying anything about it as you know. There exists a similar issue, #450, reported for h264 and media codec with the same error message. That one is also not resolved yet.

[Vigrond]

I gave up on it as I came to the conclusion ffmpeg support for MediaCodec is experimental at best. MediaCodec is meant to be implemented natively using Android API and is still relevantly new technology for Android.

It is kind of like using a wrapper for a wrapper for a wrapper situation.

Perhaps there have been some updates since, but you may have to dive into the source code for ffmpeg to see how it uses MediaCodec to understand.

[kartik1225]

In the next version of FFmpeg after v5.1 there seems to have some update regarding android media codec. MediaCodec encode is also mentioned in the change log. But not sure when next version will come out.

Here is the changelog.

[tanersener]

ffmpeg-kit v6.0 is released. I suggest checking if this issue is resolved in the new version or not.

[tanersener]

@jerrypaulsam Correct me if I'm wrong. This ffmpeg-kit issue describes a problem in mediacodec decoder, in hevc_mediacodec specifically, which prints Error while decoding stream #0:0: Try again message to the logs. ffmpeg-kit package and version is shared by the issue owner. ffmpeg command that generates that error is also given in the first post.

So, there is another issue reported in the render project. Which looks like uses ffmpeg-kit according to the logs. But ffmpeg-kit package and version information is not provided in the ticket. ffmpeg command used is also not visible to us. Logs don't include the error message shared in this issue (Error while decoding stream #0:0: Try again). They print something about OMX.qcom.video.encoder.avc, which actually is an encoder. And you say that it is the same problem and that render issue proves that this issue is not resolved in ffmpeg v6.0, right?