hls.js icon indicating copy to clipboard operation
hls.js copied to clipboard

HLS automatically downgrades the quality level on a 404 fragment error and enables autoLevelEnabled automatically.

Open vkaswin opened this issue 8 months ago • 14 comments

In live streaming, if any fragment fails with a 404 error, HLS automatically downgrades the quality and enables autoLevelEnabled.

For example, if the manifest contains four quality levels (180p, 360p, 720p, 1080p) and the player is at 1080p with autoLevelEnabled set to false, a 404 error on a fragment causes HLS to switch to 720p and enable autoLevelEnabled. If a fragment in 720p also fails with a 404 error, HLS switches to 360p, and so on. However, the player does not switch back to 1080p or 720p afterward.

How can we prevent HLS from automatically switching quality levels and avoid enabling autoLevelEnabled?

In the log below, the fragment wmslive_media_47.ts fails with a 404 error, and then HLS switches the level from 3 to 2 with the following warning. Same happens for wmslive_media_49.ts, the request fails with 404 error, and then HLS switches the level from 2 to 1.

[warn] > [warning]: switching to level 2 after fragLoadError [warn] > [warning]: switching to level 1 after fragLoadError

I have attached the logs for reference.

Hls version : 1.5.19 Hls Config : { "liveSyncDurationCount": 4, "maxBufferLength": 5, "backBufferLength": null, "maxBufferSize": null, "fragLoadingMaxRetry": 10, "manifestLoadPolicy": { "default": { "maxTimeToFirstByteMs": null, "maxLoadTimeMs": 20000, "timeoutRetry": { "maxNumRetry": 2, "retryDelayMs": 0, "maxRetryDelayMs": 0 }, "errorRetry": { "maxNumRetry": 2, "retryDelayMs": 1000, "maxRetryDelayMs": 8000 } } }, "playlistLoadPolicy": { "default": { "maxTimeToFirstByteMs": 10000, "maxLoadTimeMs": 20000, "timeoutRetry": { "maxNumRetry": 5, "retryDelayMs": 0, "maxRetryDelayMs": 0 }, "errorRetry": { "maxNumRetry": 5, "retryDelayMs": 1000, "maxRetryDelayMs": 8000 } } }, "fragLoadPolicy": { "default": { "maxTimeToFirstByteMs": 10000, "maxLoadTimeMs": 120000, "timeoutRetry": { "maxNumRetry": 10, "retryDelayMs": 0, "maxRetryDelayMs": 0 }, "errorRetry": { "maxNumRetry": 10, "retryDelayMs": 1000, "maxRetryDelayMs": 8000 } } }, "autoStartLoad": true, "abrBandWidthUpFactor": 1, "abrBandWidthFactor": 1 }

The others configurations are default value.

hls.log

vkaswin avatar Mar 10 '25 16:03 vkaswin

Could you please check and help me resolve the issue I am facing? @robwalch

vkaswin avatar Mar 12 '25 04:03 vkaswin

How can we prevent HLS from automatically switching quality levels and avoid enabling autoLevelEnabled?

This is working as expected. What do you want to happen instead? Do you want the player to error and fail playback when there are alternates available?

robwalch avatar Mar 18 '25 23:03 robwalch

We don’t want the player to switch quality on a 404 error. Currently, the player switches quality if either the fragment or the manifest returns a 404 error. Previously, we used HLS version 1.3.4, the player did not switch quality on a 404 error. However, in HLS version 1.5.19, the player switches quality when a 404 error occurs and does not switch back to the quality that previously encountered the 404 error. @robwalch

vkaswin avatar Mar 19 '25 00:03 vkaswin

With v1.5 and up, if you want to retry on 4xx errors you need to configure a custom fragLoadPolicy with a shouldRetry callback in the errorRetry block (added with #5762):

https://github.com/video-dev/hls.js/blob/master/docs/API.md#shouldretry

If all retries fail, the logic to switch will still prevail. To prevent the error-controller from resolving the error by switching, you could change the errorAction on the error event:

https://github.com/video-dev/hls.js/blob/8d8ac74862cb2b120416f924520601153294a1e6/src/controller/error-controller.ts#L127

Or, mark it resolved even when it is not to prevent switching to auto:

https://github.com/video-dev/hls.js/blob/8d8ac74862cb2b120416f924520601153294a1e6/src/controller/error-controller.ts#L497-L506

Changing the error handling after the fact won't re-enable retries. You need to configure hls.js to retry on 404 if you are serving HLS playlists that point to content that is not yet available.

robwalch avatar Mar 19 '25 01:03 robwalch

Thanks for your response. I will check it.

vkaswin avatar Mar 19 '25 02:03 vkaswin

hls-version-1.3.0.log hls-version-1.5.18.log

I tried modifying the errorAction data received in the error event, but it didn't work.

Also, I noticed that the retry handling behavior differs significantly between version 1.3.0 and 1.5.18 of HLS.js.

In version 1.3.0, if a fragment request returns a 404, and after the maximum retry attempts fail, HLS.js loads the failed fragment from the next available level. Once it is successfully loaded, it switches back to the original level.

In version 1.5.18, if a fragment request returns a 404, after the max retries, HLS.js switches to the next available level, enables autoLevel, and does not return to the original level, even if the network condition was good.

Is there any way to modify HLS.js in version 1.5.18 to handle retries the same way as in 1.3.0? The retry handling in version 1.3.0 seems much cleaner.

Additionally, I’ve observed that the retry count in the shouldRetry function is a cumulative count of fragment errors, not per-fragment. Why is the retry count cumulative instead of being tracked per fragment?

https://github.com/video-dev/hls.js/blob/44a762cf184fabb33ccf489023f3d195bc353023/src/controller/error-controller.ts#L291

I have attached the HLS.js logs for version 1.3.0 and version 1.5.18. Can you please assist me? @robwalch

hls-version-1.3.0.log

hls-version-1.5.18.log

vkaswin avatar May 26 '25 17:05 vkaswin

The latest version is v1.6.2. Use this version or later with a retry handler. Please include the config and handler in your response.

robwalch avatar May 26 '25 17:05 robwalch

Additionally, I’ve observed that the retry count in the shouldRetry function is a cumulative count of fragment errors, not per-fragment. Why is the retry count cumulative instead of being tracked per fragment?

The cumulative failure count is reset on success. Individual fragment error information would be found on the Fragment object.

robwalch avatar May 26 '25 17:05 robwalch

The latest version is v1.6.2. Use this version or later with a retry handler

I tested the same behavior in HLS.js version 1.6.2, and it behaves the same as 1.5.18, unlike the retry logic seen in 1.3.0. Please check the config and the shouldRetry handler below

HLS.js Config : { "liveSyncDurationCount": 4, "maxBufferLength": 45, "backBufferLength": 30, "maxBufferSize": 60000000, "fragLoadPolicy": { "default": { "maxTimeToFirstByteMs": 10000, "maxLoadTimeMs": 120000, "timeoutRetry": { "maxNumRetry": 2, "retryDelayMs": 0, "maxRetryDelayMs": 0 }, "errorRetry": { "maxNumRetry": 1, "retryDelayMs": 1000, "maxRetryDelayMs": 8000, "shouldRetry" : function(retryConfig, retryCount, isTimeout, response, retry) { return retryCount < retryConfig.maxNumRetry; } } } }, "autoStartLoad": true, "abrBandWidthUpFactor": 1, "abrBandWidthFactor": 1 }

The cumulative failure count is reset on success. Individual fragment error information would be found on the Fragment object.

Also, I checked the individual fragment object, and it doesn't contain a retry count. The retry count seems to be tracked globally rather than per fragmen

I have attached the debug logs for reference. @robwalch

hls-version-1.6.2.log

vkaswin avatar May 26 '25 18:05 vkaswin

Hi @vkaswin,

In your example, all network errors are configured to retry once:

"maxNumRetry": 1,
"shouldRetry" : function(retryConfig, retryCount, isTimeout, response, retry) {
  return retryCount < retryConfig.maxNumRetry;

The logs show a 404 error, a retry, another 404 error, then a switch:

[error] > 404 while loading http://127.0.0.1:5500/zvpvod_media_default_1920X1080_en_2.ts
[warn] > [stream-controller]: Fragment 2 of main 2 errored with fragLoadError, retrying loading 1/1 in 1000ms
[error] > 404 while loading http://127.0.0.1:5500/zvpvod_media_default_1920X1080_en_2.ts
[warn] > [error-controller]: switching to level 1 after fragLoadError

This is expected behavior for the configuration. You can increase the number of retries, or increase the delay between them, if your live stream produces URIs that are not available (404 - not found).

In version 1.5.18, if a fragment request returns a 404, after the max retries, HLS.js switches to the next available level, enables autoLevel, and does not return to the original level, even if the network condition was good.

A 404 indicates a problem with the server, not network conditions, so avoiding playlists that produce segment 404s will continue to be expected behavior moving forward.

Also, I checked the individual fragment object, and it doesn't contain a retry count. The retry count seems to be tracked globally rather than per fragment

Level.fragmentError is incremented on this type of error and used to penalize level selection. Level.fragmentError and Level.loadError (playlist request errors) can be reset by calling hls.stopLoad() and hls.startLoad(). Otherwise, these counts and any penalties applied, are only cleared after a segment is successfully loaded and buffered and a playlist (fragmentError) and a playlist reloaded (loadError).

Fragments that are skipped may be marked with a gap property, but otherwise, individual fragment request errors are not tracked on fragments.

How can we prevent HLS from automatically switching quality levels and avoid enabling autoLevelEnabled?

If you don't care about the retries and just want to restrict switching, then the follow solution suggested in https://github.com/video-dev/hls.js/issues/7221#issuecomment-2847642418. You can also force a switch back to the other level later if you like by setting hls.loadLevel.

  • #7221

robwalch avatar May 26 '25 22:05 robwalch

Conversation regarding this issue posted in PR: https://github.com/video-dev/hls.js/pull/7280#issuecomment-2913496418

robwalch avatar May 28 '25 03:05 robwalch

Hi @robwalch ,

Clearing Level.fragmentError counts after some period of stable playback, and easing any penalties on error, may be the best path forward for #7074.

Could you please confirm whether this will be addressed in an upcoming update?

vkaswin avatar May 28 '25 04:05 vkaswin

Hi @robwalch ,

Clearing Level.fragmentError counts after some period of stable playback, and easing any penalties on error, may be the best path forward for #7074

Could you please let me know if there is any API available in the latest version to ease penalties? If not, will this be addressed in upcoming updates?

vkaswin avatar Jun 02 '25 16:06 vkaswin

How can we prevent HLS from automatically switching quality levels and avoid enabling autoLevelEnabled?

This was solved with #7280's preserveManualLevelOnError.

Could you please check and help me resolve the issue I am facing?

The problem is that segment URIs in the HLS playlist 404. Adding a shouldRetry callback that returns true forces retries even for error codes that should not be retried.

Could you please let me know if there is any API available in the latest version to ease penalties?

Set the error count directly: hls.levels[n]. fragmentError = 0.

In version 1.3.0, if a fragment request returns a 404, and after the maximum retry attempts fail, HLS.js loads the failed fragment from the next available level. Once it is successfully loaded, it switches back to the original level. Is there any way to modify HLS.js in version 1.5.18 to handle retries the same way as in 1.3.0?

Reset Level.fragmentError to 0 and use shouldRetry for 404s that you would like your application to handle as temporary.

Could you please confirm whether this will be addressed in an upcoming update?

I have no plans to.

robwalch avatar Jun 02 '25 18:06 robwalch