track-processors-js icon indicating copy to clipboard operation
track-processors-js copied to clipboard

Adopt WebGL for virtual background when available

Open kyleparrott opened this issue 7 months ago • 1 comments

I've been migrating my app from Twilio and recently had some users reporting performance issues, which appear to be related to using virtual backgrounds in our livekit calls.

I was able to reproduce this on the track processor sample app when applying a cpu slowdown in the chrome dev tools. When using the same cpu slowdown settings on a Twilio virtual background example app I did not see the same long running tasks in performance profiles. I've attached screenshots from the profiles for the Livekit and Twilio example apps below.

The performance difference looks like it comes down to Twilio's library using a webgl pipeline to apply the virtual background when webgl is supported.

Twilio profile: image

Livekit profile: image

kyleparrott avatar Jul 08 '24 16:07 kyleparrott