rust-sdks icon indicating copy to clipboard operation
rust-sdks copied to clipboard

example: replace logo video with a screen-sharing track of the primary screen

Open daniel-abramov opened this issue 1 year ago • 4 comments

A concise follow-up on https://github.com/livekit/rust-sdks/issues/92 - I've decided to build a quick-and-dirty prototype of what it would look like if we captured a screen.

For the demo, I decided to use screenshots-rs library. It's not the fastest one, but the one that is very easy to use (I wanted to use scrap, but it has not been updated for years, and rustdesk uses a fork of scrap with their own changes that are unfortunately AGPL licensed). But since this is just for the sake of the example and is going to be replaced with a media devices API eventually, I think it's fine 🙂

I decided not to do further refactorings and improvements for now (I noticed that the example code could be simplified/streamlined a bit) to keep the PR small.


The resize() step is probably very suboptimal at this stage and I'm not sure what's the most elegant and fast solution there - when real screen-sharing is used (possibly when capturing a single window), the size of the window changes. However, the dimensions of the source are configured upfront, so I wonder if changing the size of the captured window would result in re-publishing of the track (to change the source resolution) or if the resizing step would be necessary (to make sure that the image fits into what was configured on a source level). That being said, perhaps the current resizing step could be done in a better way.

P.S.: screenshots-rs screen capturing is not super fast and given that I additionally resize() the image, I doubt that a very high FPS is achievable like this (without overloading the CPU). Looking forward for a native API to efficiently capture the screen and send it to the source 🙂

daniel-abramov avatar Oct 03 '23 17:10 daniel-abramov