mediadevices
mediadevices copied to clipboard
Unexpected too fast duration between frames when using broadcaster
With the new mediadevices
design, it relies on automated property detection by using broadcaster property, which guarantees to not drop frames up to the internal buffer size:
https://github.com/pion/mediadevices/blob/4057524bf0464879dcfe498c7fef8f1df36b9162/meta.go#L9-L21
This approach was taken so that we can support a dynamic video source, which allows users to transform their frames before being encoded. However, this approach has a drawback, it's not possible to get the first property without reading the source, this also implies that the second reader (after meta data reader in the snippet) will retrieve its first frame from the buffer. While using the buffer to get the first frame makes sure that we won't have a frame drop, it causes an unexpected too fast duration between the first frame and second frame.
Since the second reader usually reads as fast as it can, it reads the first frame, and it'll read the second frame immediately since there's no delay that's imposed by the original source.
Related issue: https://github.com/pion/mediadevices/pull/249