shaka-player
shaka-player copied to clipboard
shaka-project
A/V sync problems with unaligned HLS streams
Have you read the FAQ and checked for duplicate open issues? Yes
What version of Shaka Player are you using? v4.1.1
Can you reproduce the issue with our latest release version? Yes
Are you using the demo app or your own custom app? Demo App
What browser and OS are you using? GoogleChrome, Firefox, Ubuntu
I have a media content with video and audio encoded separately and different rates. During its streaming, sometimes when an adaptation event occurs and bitrate is changed , shaka is repeating the download and playback of the last video segment, with a different rate, that is leading video and audio out of sync.
I've experienced this behaviour on shaka player demo and on a custom application using v4.1.1 and v3.4.0 on chrome and firefox, and changing the bandwidth using the browser network tools. I also made changes in the main playlist to be similar to a hls main playlist example provided by apple at https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_ts/master.m3u8, but it didn't solve the problem.
Link to the main playlist of the encoded video: https://storage.googleapis.com/akiry-public/video-and-audio-4s/upload_41af1db0-eb71-11ec-85da-710193d27b0f.m3u8
Screenshot of shaka player demo
I can reproduce the issue. I'll take a closer look at the content. I suspect that this is a problem with your encoding, but I'll try to verify.
Okay, I understand the problem now. This would occur with any content whose video segments or audio segments are not all aligned across playlists.
For example, the first four segment lengths across 3 different streams:
| x | 360p | 720p | 1080p |
|---|---|---|---|
| seg 0 | 4.000 | 4.066 | 4.000 |
| seg 1 | 4.000 | 4.033 | 4.000 |
| seg 2 | 4.000 | 3.966 | 4.000 |
| seg 3 | 4.000 | 4.000 | 4.000 |
Switching between 360p and 1080p would be fine, but between 360p and 720p or 720p and 1080p would trigger the issue.
This appears to be a bug in StreamingEngine, which doesn't account for adaptation. On adaptation, just as on seek, we need to ask MediaSourceEngine to reset the timestamp offset, to ensure the new content is correctly placed on the timeline.
We've had to turn off ad stitching and we think this issue is the root cause. Just wanted to add my voice to the issue, and it's pretty serious for us.
Thanks. I bumped the priority and tentatively assigned the issue to @JulianDomingo. I'll take it back if Julian doesn't have the bandwidth for it.
We could probably help with providing test streams if needed (or using yospace hls cmaf reference streams should work).