Question around HLS Low latency behaviour

[luckygoyal-bitmovin]

I am testing Apple HLS-LL stream with media3/Exoplayer and have observed following behaviour which does not seem optimum. The player is requesting all video variants playlists between downloading video partial segments.

• I used Apple HLS-LL reference stream https://ll-hls-test.cdn-apple.com/llhls4/ll-hls-test-04/multi.m3u8 for testing

• It has 4 video variants. I captured network traffic of playlist and segment requests using Charles proxy and observed that player is making request for all 4 video variant playlists between downloading consecutive partial video segments. Charles file is attached as zip. HLS_LL_Apple_multiple_variant_download.chls.zip

• The behaviour seems sub-optimal as it will increase load on CDN and would may contribute to increased latency. This test stream has only 4 video variants but a production stream could have 6-8 video variants making the behaviour worse if all 8 video variants are downloaded between each consecutive partial segment download.

• Sharing a screenshot of Charles capture showing all variant playlists downloaded between segment downloads

Questions

• Is this expected behaviour of Exoplayer? If yes, then what is the rational behind this?
• Is there a configuration to change the behaviour to only download the active variant playlist from which the partial segments are picked for download?

[luckygoyal-bitmovin]

Dear Google Team, is there any feedback on this? Please do let me know if any more information is required.

[tianyif]

Hi @luckygoyal-bitmovin,

We're looking at this, will update this shortly.

[tianyif]

Hi @luckygoyal-bitmovin,

Thanks for the reporting! This issue is reproducible from our end. In fact, the reloads of non-primary playlists are triggered from this piece of code, and the purpose of doing that is to check if the segments hinted for preload before have been actually published, otherwise we cancel the loading of it. However, keep reloading those playlists every time doesn't sound ideal to me.

We will keep investigating and provide a solution to this.

[luckygoyal-bitmovin]

Thanks @tianyif for investigating this and sharing the update. One of our customers plan to start using HLS Low latency playback in production soon and is currently testing the same, so we are eager to have a solution for this as soon as possible. Kindly keep us updated with the progress on this investigation and if you have any solution suggestions in the interim, that will be really appreciated.

Please do also let me know if I can help in anyway. Thanks.

[luckygoyal-bitmovin]

Hi @tianyif , hope you are doing well. Is there any further update about fix for this behaviour?

[mayankmediakind]

hey @tianyif : Can you please help provide an update on this one, this behavior is adding to the latency on when trying to playback HLS low latency content. This is important for us to get fixed to get this into production, Can you please take a look at this one on priority. cc: @rohitjoins

[tianyif]

Hi @luckygoyal-bitmovin and @mayankmediakind,

We're working on a fix for this issue, and we need to make sure that the comments in the internal review cycle are addressed until it is available on Github. Thanks!

[luckygoyal-bitmovin]

Thanks for the update @tianyif . We will be eagerly waiting for the fix.

[tianyif]

Hello @luckygoyal-bitmovin,

We just submitted the fix to the main branch, while we did the manual testing with Apple LL-HLS sample stream https://ll-hls-test.cdn-apple.com/llhls4/ll-hls-test-04/multi.m3u8, we encourage you to do more tests with different streams. Please let us know if this improves or causes any regression.

[luckygoyal-bitmovin]

Hi @tianyif , thanks a lot for the fix. I will test it today and will share my observations.

[luckygoyal-bitmovin]

Hi @tianyif , I did some initial testing. With the fix, the requests for non-primary video playlists is not sent so the main issue is fixed. But I observed following side effects as well.

• For video playlist, I can see that some of the request are without the LL-HLS server control query params. It seems that with newly added onPlaylistRefreshRequired(lastMediaChunk.playlistUrl) call in case if CHUNK_PUBLICATION_STATE_PRELOAD simply passes the playlist URL and does not add the required query parameters as was done in the original logic loadPlaylistInternal(getMediaPlaylistUriForReload()). Without server control query parameters HLS-LL server will not be able to respect features like blocking playlist reload, delta updates etc.

• For audio playlists all the requests are without server control query parameters.

• Another difference in behaviour is that audio playlist is now refreshed/downloaded at much less frequency now. It seems audio playlist is requested only when the pre-load segment in the previous downloaded playlist is requested. Before this change, audio playlist was refreshed/downloaded after every part segment download. Not sure if this is intended behaviour but lower frequency of playlists download prevents player from knowing about the latest available parts as soon as these are available.

[tianyif]

Thanks @luckygoyal-bitmovin for the observation!

The bullet points 1) and 2) are related. For now, in the implementation of DefaultHlsPlaylistTracker.refreshPlaylist, it calls the MediaPlaylistBundle.loadPlaylist() with the a plain playlist url (no server control query params). A thought with me is whether we should also use MediaPlaylistBundle.getMediaplaylistUriForReload() in this place instead, as the MediaPlaylistBundle keeps the status of the snapshots of the playlist, so it should be able to produce the correct url at any point. I will discuss with the team if this is something missing, or this was left on purpose, and then decide a solution to accommodate this case.

And the bullet point 2) and 3) are related on another perspective. In fact, the audio playlist is also refreshed when the HlsSampleStreamWrapper tries to continue loading but there is no segment or part available in the current playlist, via onPlaylistRefreshRequired. With this change, I saw this part of code is hit more frequently than before. The reason being this is we only allow the primary playlist (while audio playlist is not) to be refreshed periodically in this change, then the next audio chunks are more likely to be unavailable when we attempt to load it, then the HlsSampleStreamWrapper demands the refresh on the playlist. It looks like we refresh the audio playlist for non-LL live stream in this way. Again, I'll also need to check with the team if this is on purpose.

As before, I will update in the comment once the above two points are addressed.

[luckygoyal-bitmovin]

Hi @tianyif , thanks for the feedback. Will look forward to further updates.

[luckygoyal-bitmovin]

Hi @tianyif , hope you are well. Just checking in if there is any further update on the this issue?

[tianyif]

Hello @luckygoyal-bitmovin,

We have a commit merged recently for above follow up issue 1) - https://github.com/androidx/media/commit/e180e263a59e95d623185097e50a658c0a6826e2 (looks like there is no tag in the commit linked this issue - sorry!), we're now working on the follow up issue 2).

[luckygoyal-bitmovin]

Thanks @tianyif for the update. Looking forward to fix for issue 2 as well.

[luckygoyal-bitmovin]

Hi @tianyif , hope you are well. Just checking in if there is any further update on the fix for follow up issue 2?

[luckygoyal-bitmovin]

Hi @tianyif , just checking in if there is any further update? As we are getting closer to using LL-HLS in production, this issue is becoming more critical for successful deployment.

[tianyif]

Hi @luckygoyal-bitmovin,

Sorry for the delay! Been working on the other stuffs earlier, but I just sent a fix for internal review. Will try to make it to the upcoming beta01 release.

[luckygoyal-bitmovin]

Hi @tianyif , thanks for the update and making progress on the fix. I will look forward to test the fix and share my feedback.

[tianyif]

Hi @luckygoyal-bitmovin, do you have a sample HLS live stream with multiple audio tracks that can be provided to us to test?

[luckygoyal-bitmovin]

Hi @tianyif , the LL-HLS test stream I have at hand is having one audio track. I am checking around if I can find one with multiple audio tracks. Will update here latest by Monday if I found one.

[luckygoyal-bitmovin]

Hi @tianyif , I have sent an email to [email protected] containing a HLS-LL test stream URL with mulitple audio tracks. Kindly let me know if this is sufficient for your testing.

[tianyif]

Hi @luckygoyal-bitmovin,

Thanks for the stream URL - it is sufficient for testing! We recently pushed the fix to the development branch, and it should be available in the upcoming 1.4.0-beta01 release.

[luckygoyal-bitmovin]

Hi @tianyif , thanks for the update and all the help with fixing this issue. We are planning to cherry pick the commits containing the fix for this issue. As per my understanding, following 3 commits compose the fix for the issue. Can you please confirm if cherry picking these 3 commits should be sufficient or are there any other code commits to be picked up?

• 50fefe6
• e180e26
• 86a60e6

[tianyif]

That's true - if you're cherry-picking them. And also they will be all available in 1.4.0-beta01 release.

[luckygoyal-bitmovin]

Thanks for the quick response.

[luckygoyal-bitmovin]

Hi @tianyif , wanted to update you that the fix is working as expected in our initial tests, so I am closing this ticket. Thanks for all the support in fixing this issue.