ffmpeginteropx
Problem with new HW acceleration and stream buffering
If we want to make our new native HW acceleration the new default, we will have a problem with streaming. The problem is this: Currently, we rely on the MSS buffer to get stable playback in streaming scenarios. The MSS buffer works by buffering our output samples. With the old HW acceleration (passthrough to system decoder), the output are compressed samples in system memory, so that works reasonably well (besides some general bugs in MSS buffering). But with the new HW acceleration, we output decoded frames in GPU memory. Using a buffer on that will quickly blow up GPU memory usage.
If we want to make our approach work with streaming as well, we'd have to provide our own internal buffering implementation.
One approach how this could be done: Instead of reading packets on demand, we could have an async read-ahead packet reader. It would read packets and fill stream buffers until a certain time limit or binary size limit is exceeded. Instead of having streams pull packets on demand, they have to wait until packets are made available.
I wonder if we could get the OS to do the buffering for us, by loading ahead in the stream.
The problem is that FFmpeg does all the stream creation and handling. And none of the buffering options seem to do any effect. There is no general read-ahead mechanism in FFmpeg.
This sounds like a consumer-producer problem.
Take a look at this, looks promising...
https://github.com/cameron314/concurrentqueue
Unfortunately I won't have much time to look into this in the following weeks...
In the mean time we could do a work around and force system codes when we create a stream source from URL.
Same for me. I will be very busy the next 5-6 weeks. Been building a house since summer 2020, and it's almost finished now! ^^
Thank you. Really looking forward to moving in, maybe in March!! :D But first we need to do the painting and floors ourselves, that will keep us busy for the next weeks...