Seamless transitions to clipped items with a second decoder

[GlebPBanuba]

Version

Media3 main branch

More version details

No response

Devices that reproduce the issue

All

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

We are experiencing the serious issue while playing video by trimmed media sources within single video. Let's say there is a video (60 seconds) and the user wants to trim the video by selecting a number of sources in the video i.e.

1. 500 ms - 2000ms
2. 4000ms - 3000ms
3. 6200 ms - 7950ms
4. ... N. 55000ms - 59000ms

The player does not play these source smoothly while using ConcatenatingMediaSource and ConcatenatingMediaSource2.

This sample https://github.com/Banuba/android-media3-issue-concatenating is prepared to clearly demonstrate the issue.

PTAL and advice how to mitigate an impact of this issue.

Expected result

No long visible freezes

Actual result

Long freezes while playing a number of trimmed media source within one video

Media

This sample https://github.com/Banuba/android-media3-issue-concatenating is prepared to clearly demonstrate the issue. Zip file with the sample is attached as well. android-media3-issue-concatenating.zip android-media3-issue-concatenating.zip

Bug Report

• [X] You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

[tonihei]

Thanks for sharing the example project!

The reason for what you are seeing is that each item is played independently. Each individual clip needs to decode samples from the previous keyframe to start at the correct frame. By default, ExoPlayer uses a single codec for a playback, so at every transition there is small gap where the codec needs to process all the samples from the previous keyframe that are not shown on screen.

We are working on improvements that allow to use a second decoder for these types of transitions. This way the second decoder can already decode the required samples for the next clip and the transitions can become seamless again. This is still work in progress though, so I'll mark it an enhancement. @microkatz not sure if there is already another enhancement tracking this support, feel free to de-duplicate if so.

[microkatz]

@GlebPBanuba

The pre-warming feature has been launched in the main branch. If you are not using a custom video renderer, you can enable pre-warming via the simple experimentalSetEnableMediaCodecVideoRendererPrewarming api. This should reduce the latency and allow seamless playback for your short, clipped videos.

Hope this helps and works for you!

[microkatz]

@GlebPBanuba

I'm going to close this thread as there has not been a response to the last comment about the new feature addition. Hope the pre-warming feature works for your use case!

Please open up a new ticket if you have any issues or additional questions.