udevbe
Use WebTransport instead of WebSocket
Compositor-proxy currently uses WebSockets to communicate application [video] frame data. This has well known drawbacks for use in real-time communication as it's essentially a TCP connection. Instead we should use WebTransport which offers UDP semantics which is far better suited for real-time communication.
Initial preparing work has already been completed by switching to the uwebsocketjs library which will eventually support WebTransport
fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.
So we patiently await the support and arrival of WebTransport.
https://caniuse.com/webtransport
https://caniuse.com/webcodecs
All major browsers other than Safari now support WebTransport. Firefox is working on WebCodecs.
https://github.com/selkies-project/selkies-gstreamer/issues/48#issuecomment-1764124875
We've been investigating WebTransport but there are a few catches outlined above (requiring valid HTTPS certificates and other issues).
But overall, the web browser stack is ready.
More references: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1571 https://github.com/kixelated/moq-rs https://github.com/wanjohiryan/warp https://github.com/wanjohiryan/Arc3dia
fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.
We (https://github.com/selkies-project/selkies-gstreamer) use WebRTC DataChannels for passing user input and clipboard, and use the native WebRTC MediaStreams for video and video. It's relatively inflexible but more usable than using DataChannels for everything.