hls.js icon indicating copy to clipboard operation
hls.js copied to clipboard
verified publisher icon video-dev

Interstitial asset scheduling accuracy

Open bbert opened this issue 6 months ago • 3 comments

Is your feature request related to a problem? Please describe.

Scheduling interstitial assets loading and playing is currently triggered by the timeupdate event of the video element, which leads to inaccurate scheduling since timeupdate event is triggered at most every few hundred ms.

For scenario such as ad replacement accuracy is critical for user experience.

Describe the solution you'd like

A solution could be to rely on the requestVideoFrameCallback() API, if available, to be frame accurate.

If requestVideoFrameCallback() API is not available, a fallback solution could be to set a timer as we are getting close to the intersitial time, without waiting for next behindhand timeupdate event.

Additional context

No response

bbert avatar Aug 19 '25 08:08 bbert

HLS.js uses TextTrack cue "enter" events to improve the timing of interstitial scheduling, which is significantly more accurate than timeupdate.

robwalch avatar Aug 19 '25 15:08 robwalch

We could add support for requestVideoFrameCallback and/or requestAnimationFrame to try to improve timing of interstitial schedule changes, but both would be less performant, and neither would prevent rendering of buffered content past DateRange start times as the event loop execution is not strictly bound to the rendering pipeline.

In your testing, please try observing the timing of cue "enter" events: https://github.com/video-dev/hls.js/blob/b21b6de9f67b02ded0e6dcb6330785ad168bc8e0/src/controller/id3-track-controller.ts#L497-L499

https://github.com/video-dev/hls.js/blob/b21b6de9f67b02ded0e6dcb6330785ad168bc8e0/src/controller/interstitials-controller.ts#L972-L973

  1. Does the callback execute when you expect - where on the timeline and with what frame rendered?
  2. Does the interstitial-controller update the schedule in this callback stack (as opposed to waiting for currentTime to advance further in an upcoming "timeupdate" event)?
  3. Does the resulting media element transition immediately interrupt primary playback, or is there additional delay in tearing down the rendering pipeline after the schedule change is initiated?
  4. Please list results for each browser or device that you are focusing testing on.

Thanks!

robwalch avatar Aug 19 '25 17:08 robwalch

Thanks @robwalch for your answer. My bad, there was a misconfiguration in the used test (on-demand) stream (PROGRAM-DATE-TIME set to 1/1/1970) which led to missing cue events. Indeed, cue events seems to be accurate enough and requestVideoFrameCallback may not improve that much. Nevertheless I notice difference between the metadata.mediaTime of the requestVideoFrameCallback and the <video>.currentTime.

For example, on latest chrome (139.0.7258.139) on windows desktop, with an on-demand stream and interstitial event starting at second 5.00:

with triggering of interstitial based on cue event:

### timeupdate 4.708224
### videoframe mediaTime: 4.723222  currentTime 4.716432 => requestVideoFrameCallback() handler
### videoframe mediaTime: 4.7399  currentTime 4.73443
### videoframe mediaTime: 4.756566  currentTime 4.750046
### videoframe mediaTime: 4.773222  currentTime 4.766443
### videoframe mediaTime: 4.7899  currentTime 4.783305
### videoframe mediaTime: 4.806555  currentTime 4.799951
### videoframe mediaTime: 4.823233  currentTime 4.816629
### videoframe mediaTime: 4.8399  currentTime 4.83316
### videoframe mediaTime: 4.856566  currentTime 4.849771
### videoframe mediaTime: 4.873222  currentTime 4.866674
### videoframe mediaTime: 4.8899  currentTime 4.883199
### videoframe mediaTime: 4.906566  currentTime 4.899984
### videoframe mediaTime: 4.923222  currentTime 4.91672
### videoframe mediaTime: 4.9399  currentTime 4.933299
### videoframe mediaTime: 4.956555  currentTime 4.950235
### videoframe mediaTime: 4.973222  currentTime 4.966485
### timeupdate 4.974206
### videoframe mediaTime: 4.9899  currentTime 4.983294
### videoframe mediaTime: 5.006566  currentTime 4.99988
### cueenter currentTime:5.006017
### timeupdate 5.006017
[log] > [interstitials]: setSchedulePosition 1, undefined (["ad": 5.000-32.279])
[log] > [interstitials]: INTERSTITIAL_STARTED ["ad": 5.000-32.279]

and with triggering of interstitial based on requestVideoFrameCallback event:

### videoframe mediaTime: 4.7399  currentTime 4.72169    => requestVideoFrameCallback() handler
### timeupdate 4.722676
### videoframe mediaTime: 4.756566  currentTime 4.739691
### videoframe mediaTime: 4.773222  currentTime 4.755592
### videoframe mediaTime: 4.7899  currentTime 4.771638
### videoframe mediaTime: 4.806555  currentTime 4.788413
### videoframe mediaTime: 4.823233  currentTime 4.804963
### videoframe mediaTime: 4.8399  currentTime 4.821726
### videoframe mediaTime: 4.856566  currentTime 4.838499
### videoframe mediaTime: 4.873222  currentTime 4.855021
### videoframe mediaTime: 4.8899  currentTime 4.87178
### videoframe mediaTime: 4.906566  currentTime 4.888557
### videoframe mediaTime: 4.923222  currentTime 4.904907
### videoframe mediaTime: 4.9399  currentTime 4.921656
### videoframe mediaTime: 4.956555  currentTime 4.938606
### videoframe mediaTime: 4.973222  currentTime 4.954937
### videoframe mediaTime: 4.9899  currentTime 4.971851
### timeupdate 4.985799
### videoframe mediaTime: 5.006566  currentTime 4.989068
[log] > [interstitials]: setSchedulePosition 1, undefined (["ad": 5.000-32.279])
[log] > [interstitials]: INTERSTITIAL_STARTED ["ad": 5.000-32.279]

bbert avatar Aug 20 '25 09:08 bbert