Better performance in bigger video Width

[drenmullafetahu]

Very useful integration. Congrats on your implementation.

My question is: Is there a way to increase the performance in bigger video size. I saw that right now it uses 640px video size. When you resize the video to 1980px the performance drops drastically.

So is there a way to bypass this issue . Thanks in advance.

[Volcomix]

Yeah, even the 1280×720 video of this demo get a performance drop when resizing through WebGL. I tried to mitigate this by downloading the pixels asynchronously after resizing on GPU but uploading video textures to GPU with WebGL is also pretty slow. Maybe there is a way to improve it by mixing 2D Canvas for resizing and WebGL for post-processing only.

[benbro]

Commercial sdk that improves the performance. Not sure how hard to implement.

[benbro]

Can this improve performance for large videos with the WebGL2 pipeline?

1. Add a new sourceCanvas with the size of the mask.
2. Resize by copying from sourcePlayback.html to sourceCanvas with a 2d context.
3. Use sourceCanvas instead of sourcePlayback.html when calling gl.texImage2D here:
4. https://github.com/Volcomix/virtual-background/blob/main/src/pipelines/webgl2/webgl2Pipeline.ts#L170

[IDDD-Dennis]

Did anyone figure a way to increase the quality while keeping performance?