ExoPlayer not able to achieve Target Latency

[nuwins-purdue]

Hi, We are testing ExoPlayer with our hosted low latency stream. We are also comparing it with the industry-standard Dash.Js player. We observe that ExoPlayer cannot catch the target latency and always stays behind for 2-3 seconds. At the same time, the Dash player is able to catch up with the target latency. I am attaching the snapshot of the hosted stream. The stream can also be shared through email. MyDash.txt

[imranbuet63]

We wanted to know what can be tweaked on the player side/stream for the player to achieve better latency. Also some explanation of what is going wrong with the current setup will be very helpful.

[marcbaechinger]

I assume @nuwins-purdue and @imranbuet63 are both reporting separate issues.

For looking into any of your issues, we would need more information to understand the issue and making it actionable for us. So when asking about what is going wrong with the current setup, we would need to know what current setup.

a) Best would be a test stream so we can look into the behaviour of the player for these streams. This procedure is very likely to receive a good response based on the stream and the actual issue. If you're unable to share test content publicly, please send them to [email protected] using a subject in the format "Issue #10645". Please also update this issue to indicate you’ve done this.

b) Alternatively, we would probably need an explanation of the expected behaviour and what this expected behaviour is based on (like manifest elements and attributes as well as the physical segment structure in the stream). For a DASH stream that would be you explaining to have this and that 'serviceDescription` element that configures some latency behaviour that is this and that. Then we would need to have some insight in what is not working as expected, this would then be the actual latency that you measure and how you measure and how this differs from the expectation and what you see with other players.

It's obviously easier for us to test this with the stream (option a)), because we can then see how for instance a given DASH manifest is reflected in the LL live configuration of the player, whether this configuration meets the expectation of us and the stream publisher and whether the player is able to achieve the target latency as expressed by that configuration. Approach b) is difficult because it needs an explanation and understanding of what is going on and even more if we don't have a similar live stream ourselves for testing.

Regarding configuration options, please read the DevGuide about live streams and then start asking for details based on the knowledge achieved there.

[imranbuet63]

@marcbaechinger I shared the stream over email. It can be accessed from any network. You can tag my account @imranbuet63 for further communications. We noticed two behaviors:

1. We are logging the targetOffsetMs on the player and noticed that it never reaches 3s latency which is configured on the mpd.
2. We also noticed that the player starts with the highest resolution 1920x1080 and immediately drops to the lowest one 480x270 and never recovers.

Both of these behaviors are observed across many runs we did. We would like some insights on them. Thank you.

[marcbaechinger]

@imranbuet63 Thanks!

I think from the manifest that you sent to our mailbox, it looks like that you are having the ServiceDescription element in the manfiest set to something like this:

<ServiceDescription id="0">
    <Latency target="3000" referenceId="7"/>
</ServiceDescription>

This makes the max and min playback speed set to C.RATE_UNSET. With this the playback speed is always set to 1 which is unit speed. When we join a live stream we can not start at the position that would be nowUs - targetLatencyOffsetUs. Instead we start at the start of the last possible segment behind the calculated target latency position. This is for various reasons, for instance because we need to start at the key frame closest before the desired target latency position.

So the player always starts behind the desired 3000 ms of target latency and then catches up by playing faster then playbackSpeed=1. With the configuration of the stream you are switching of catching up because you don't allow to change the playback speed.

If you can't change the manifest, you can consult the DeveloperGuide that explains how you can override the live stream configuration values of the manifest.

Can you try configuring this that way and then report if you see that the player catches up after having initially started? You should see the player catching up after the initial latency being larger than the target latency, until it is about the target latency. I would be curious to see your measurements of targetMsOffsetMs before and after this change. If you don't want to send the data here in public, send it per email.

[imranbuet63]

@marcbaechinger We tried this:

MediaSourceFactory mediaSourceFactory = new DefaultMediaSourceFactory(dataSourceFactory) .setLiveMinSpeed(0.99f) .setLiveMaxSpeed(1.09f) .setAdsLoaderProvider(this::getAdsLoader) .setAdViewProvider(playerView);

The results are still not satisfactory. The targetMsOffsetMs starts with ~5 sec and takes around 1 min to reach ~3.8 second which is the best we could observe. The performance is even worse over 5G compared to Wi-Fi. Sharing both the results over email.

[imranbuet63]

@marcbaechinger We tried a couple of things and we have the questions below:

1. We tried to observe the player behavior with one of the publicly hosted streams on Akamai server. We noticed that the player always starts playback with the player.getCurrentLiveOffset() == 4+ seconds. Even though the manifest sets the target latency == 3 sec. The player then takes around a minute to catch up to the latency (even if we set:

          .setLiveMaxOffsetMs(4000)
          .setLiveMinOffsetMs(1000)
          .setLiveMinSpeed(0.99f)
          .setLiveMaxSpeed(1.04f)
   

at the player).

What is the reason behind playback starting with 4+ sec of latency?

2. In our manifest (attached to the issue) you can see that the minBufferTime is set to 2sec. We currently have no way to control or modify this on the manifest so we tried on the player side. We observed that the DashManifestParser.java reads the manifest and we tried to set it there. However, noticed no improvement on the latency. Would you explain how this parameter is influencing the player's behavior and how it is used?

3. We also tried to modify the following parameters:

public static final int DEFAULT_MIN_BUFFER_MS = 1_000; ///initial value 50_000 public static final int DEFAULT_MAX_BUFFER_MS = 2_000; ///initial value 50_000 public static final int DEFAULT_BUFFER_FOR_PLAYBACK_MS = 1000; //initial value 2500 public static final int DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS = 1000; //initial value 2500

we noticed that this does solve the random bitrate switching throughout playback and solves the media load errors. In some runs, the player is able to catch up to the target latency of 3sec. However, in some runs, it still starts with 4+ sec of latency and is able to achieve only up to 3.8sec of latency. Would you be able to explain how these parameters are influencing the player's behavior in terms of low latency playback?

4. Also since we provided the manifest snapshot, do you see any problem with the manifest? We tested the stream with the Dash.Js player and it seems to play the stream fine.