Render frames concurrently

[vertify-mldev1]

Hello! As far as i can see, it is not possible to render multiple frames concurrently when transcoding a video. Is it on the roadmap to support concurrency? Or did anyone implement anything in this direction? Thanks for your help.

[izzytwosheds]

Hi. Can you clarify what is meant by concurrent frame rendering? Do you mean transcoding different sets of I and B frames? For example, keyframe 1 and following B frames would be transcoded at the same time as keyframe 2 and its following B frames. To be honest, this is not something on the road map. We didn't have a use case for that so far, and introducing concurrency would complicated transcoding logic quite a bit. I don't know if anyone out there is doing this. I can see this being more relevant to transcoding on a backend, since concurrency is more relevant and doable there. On mobile devices, hardware capacity is limited and transcoding might fail if we hit the limit.

[vertify-mldev1]

Oh, let me clarify my use case a bit:

I send the decoded frames into a mediapipe graph (which is able to process multiple frames concurrently). The output of the graph is fed to the encoder.

To achieve this i implemented a custom LiTr Renderer and noticed that renderFrame gets called serially, thus there can only be one frame in the graph at a time.

To feed multiple frames into the graph i would need to have access to multiple decoded frames at once.