Segments not being loaded on time during backward seeking with speed x16

[Mikromobi]

ExoPlayer version:

1.2.0

Issue:

I use trick mode track on a DASH stream and i try to implement backward seeking with speed -x16. As we can't use negative values in player?.setPlaybackSpeed(..), I perform microseeks (with player?.seekTo) to move backward in stream.

I need speed -x16, so each microseek lasts 1s/16 = 62.5ms and changes position by -1000ms.

My code looks like:

repeat(10000){
  player?.seekTo(player?.currentPosition-1000)
  delay(1000/16)
}

Unfortunately, it doesn't work as expected. From user's point of view the stream is blocked and frames don't change. Using AnaltyicsListener and onLoadStarted/onLoadCompleted methods I noticed that we lose segments at this speed - new chunks are not being loaded on time and onLoadCompleted is not invoked.

What's interesting:

I've already implemented forward seeking with speed x16 (using player?.setPlaybackSpeed(16)). It works fluent. When I add backbuffer using :

val loadControl = DefaultLoadControl.Builder().setBackBuffer(200_000, true)

and than I perform forward seeking (x16) to have some data in it, and after that I do backward seeking (-x16) - backward seeking works pretty well. However looks like it works well as long as we have in backbuffer data received during forward seeking. If we try to further seek backward (to seek in part of stream in which we haven't been during forward seeking) - it stops working.

Question

How to load chunks on time/cache chunks correctly during microseeks in case of backward seeking? As it's possible for forward seeking (x16) I belive it should be achievable also for backward seeking (-x16).

[tonihei]

Videos can't easily play backwards because of the way they are encoded. We always have to start from the previous keyframe and decode forwards to reach the target frame. Your attempt to play backwards already does that quite explicitly: every seek causes the player to start decoding from the previous keyframe until it reaches the specified target time. The main problem is that this process is likely too slow to show 16x speed up. In fact, it's likely too slow to even play at 1x backwards.

When I add backbuffer

This is definitely a good idea to at least make sure all the encoded data is already available in memory. If you haven't already done it, you can also use caching to avoid accessing network resources repeatedly if the back buffer isn't large enough, see https://developer.android.com/media/media3/exoplayer/network-stacks#caching

From user's point of view the stream is blocked and frames don't change

It's likely that you start a new seek before the previous seek produced a visible frame. In this case, all pending tasks are cancelled and we start again for the new seek. If none of them are fast enough, you basically never see a frame displayed. You can solve this on a best effort basis by waiting for Player.Lister.onRenderedFirstFrame before calling player.seekTo again. This way you may have to skip or delay some of the seeks, but you should be able to see some frames at least. But still, it won't be as smooth as forward playback that can simply continue decoding without ever loading data again.

[AGasztold]

Hello, thank you for your answer. Will the situation be that difficult in case of trickplay video path (1 FPS)?

[tonihei]

Will the situation be that difficult in case of trickplay video path (1 FPS)?

This should be easier because all frames are "keyframes" and there is just one frame per second. Unfortunately, there is still no built-in way to simply play backwards, but the manual approach you explained above (+waiting for the frame/image to be rendered) should work fine I guess.

[tonihei]

We are working on a new scrub mode that should address issues like these. @icbaker Not sure if there are other tracking issues already, but assigning to you for now.

[icbaker]

Scrubbing mode is available in 1.8.0, please try it out (note that enabling it doesn't change LoadControl behaviour, so you should still enable a backbuffer as described above). You can configure scrubbing mode behaviour as well.