libdragon icon indicating copy to clipboard operation
libdragon copied to clipboard

mixer: add mixer_throttle

Open rasky opened this issue 3 years ago • 1 comments

This function allows to throttle the mixer to a specific pace, to help implementing correct syncing with a video.

Normally, once the mixer is initiated and assuming mixer_poll is called frequently enough, the audio will playback uninterrupted, irrespective of any slow down in the main loop. This is the expected behavior for background music for instance, but it does not work for video players or cut-scenes in which the music must be perfectly synchronized with the video. If the video happens to slowdown, the music will desynchronize.

mixer_throttle sets a budget of samples that the mixer is allowed to generate. Every time the function is called, the specified number of samples is added to the budget. Every time the mixer playbacks the channel, the budget is decreased. If the budget reaches zero, the mixer will automatically pause playback until the budget is increased again, possibly creating audio cracks.

rasky avatar Jan 18 '22 23:01 rasky

I'm experimenting a bit with audio/video syncing. This PR works correctly in allowing a videoplayer to keep sync with audio even when it slows down. I would say in general the pull-based / callback-based approach of the mixer is not the best for a videoplayer; in a muxed video, the videoplayer would probably want to decode and push audio into the mixer at its own pace, rather than setting up for the being called back at "random" moments for audio decoding. OTOH, this works good if video and audio are in separate files, which also means that it makes it easier to use the same decoder for both videoplayers and normal background music.

I'll keep this PR in draft for now.

rasky avatar Jan 23 '22 10:01 rasky