emscripten

emscripten copied to clipboard

Feature Request: Integration of the WebTransport API into Emscripten

Overview:

WebTransport, a new HTTP/3 web standard, is already accessible in modern browsers such as Chrome, Edge, and Firefox (see compatibility at: caniuse.com). As a direct user and advocate for this feature, I strongly believe that integrating WebTransport into Emscripten should be a high priority.

Significance in Real-Time Multiplayer Games:

In the realm of real-time multiplayer games, the necessity for unreliable messaging is paramount for scalability. Consider scenarios like player movement, which demands rollback mechanisms to conceal latency issues. This underscores the critical need for unreliable messaging in web-compatible engines. My project, Toxoid, a multiplayer polyglot WASM Rust game engine that utilizes C libraries and targets the web via Emscripten, serves as a prime example of this requirement. More about Toxoid can be found here: Toxoid GitHub Repository.

Current Landscape:

For leveraging C/C++ libraries, and for Rust projects which include C/C++ libraries, Emscripten is often the go-to framework for browser/web API interoperability. However, because Emscripten does not support the WebTransport API, this indicates that it's falling behind the tier 1 Rust WebAssembly target in supporting the HTTP/3 standard.

In contrast, the Rust wasm target wasm32-unknown-unknown already supports the WebTransport API through the web-sys library (details at: wasm-bindgen), but it does not facilitate the use of C libraries.

Limitations of Alternatives:

WebSockets, operating over TCP, present substantial limitations for our use case.

TCP Network Congestion and Limitations in Real-Time Applications:

Congestion and Packet Loss: TCP, the traditional protocol used for reliable communication, often encounters significant challenges in the context of real-time applications. One of the primary issues is network congestion. As TCP ensures data integrity and order, packet loss can lead to considerable delays. The TCP protocol requires acknowledgment of packet receipt, and lost packets result in retransmissions. This process, while ensuring data integrity, increases the round trip time (RTT), negatively impacting real-time interactions.

Infinite Retries and Latency: Another critical aspect of TCP is its approach to handling message transmission failures. TCP will attempt infinite retries to send messages until they are successfully delivered. This feature, while useful for ensuring complete data transfer in many scenarios, becomes a hindrance in real-time applications. In scenarios like multiplayer gaming or live streaming, where timely delivery is more crucial than completeness, these retries contribute to unacceptable latency.

Advantages of SCTP over TCP: Stream Control Transmission Protocol (SCTP), which operates over UDP, offers a more suitable alternative for such real-time applications. Unlike TCP, SCTP (and by extension, UDP) does not guarantee packet order or reliability. This lack of strict reliability, paradoxically, is advantageous in real-time contexts. It allows for faster data transmission without the burden of handling packet loss or order, significantly reducing latency. SCTP also enables multiplexing of several streams within a single connection, mitigating the head-of-line blocking problem inherent in TCP.

The Need for a Modern Solution in Emscripten:

Given these limitations of TCP in the realm of real-time web applications, the integration of a more advanced and suitable protocol like WebTransport, which leverages the benefits of protocols like SCTP over UDP, becomes increasingly critical in Emscripten. This integration will address the challenges posed by TCP, offering a more efficient, reliable, and latency-sensitive solution suitable for real-time applications like multiplayer games.

The need for WebTransport (QUIC):

While WebRTC can facilitate unreliable messaging, it introduces significant complexity in setup. WebTransport, utilizing QUIC as its transport layer, offers a more streamlined alternative with several benefits, including multiplexing without head-of-line blocking, reduced connection establishment time, built-in security with TLS 1.3, improved congestion control, connection migration, and suitability for modern internet applications.

Proposed Technical Requirements for WebTransport Integration in Emscripten:

1. API Compatibility: Ensure the WebTransport API is fully compatible with Emscripten, allowing seamless integration and usage within the Emscripten environment.

2. Ease of Use: Provide clear, comprehensive documentation and examples to facilitate easy implementation for developers.

3. Cross-Platform Support: Ensure that the implementation is cross-platform, allowing developers to target a wide range of devices and browsers.

Do you think it will be possible to implement most of this withing the existing WebSockets APIs that emscripten has, or would it need to be completely new API?

From the MDN page is sounds like this is kind of an extension of the existing WebSockets: "The WebTransport API provides a modern update to WebSockets".. so maybe we can do this with minimal user-visible change?

@sbc100 If you were talking about POSIX sockets, then perhaps?

But if you're talking about the WebSocket API, IMHO I feel like there's a few functions like reliability(), datagrams(), congestion_control(), that are not available in WebSockets. I imagine a major benefit of WebTransport is that it allows finer grain control over networking.

Unless you were creating some general abstraction layer and was very explicit about that, I feel it'd be quite confusing to extend the WebSockets API with WebTransport functionality. From the perspective of a new user, I personally would be going "where is the WebTransport API", at least initially until reading further into some documentation that mentions this. It seems intuitively confusing. I would feel the same way if it were done this way for WebRTC.

I was talking about WebSockets. Emscripten already an API for using WebSockets: https://emscripten.org/docs/porting/networking.html.

I'm not saying that the WebTransport should or should not try to build on this layer, but its something that we should consider. Whoever takes on this task should certainly have a good understanding of all the current networking APIs in emscripten and how they overlap.

I'm interested in this as well and had looked a while back to see if anyone was standardizing on a C API for WebTransport and hadn't found anything yet.

(I stubbed out a crate for adding a C API to the Rust wtransport library, but haven't actually written any of the FFI yet as I hadn't found someone else doing similar yet. I thought it would be ideal if there were a native library with the same API as whatever happens in Emscripten.)

I'm interested in this as well and had looked a while back to see if anyone was standardizing on a C API for WebTransport and hadn't found anything yet.

(I stubbed out a crate for adding a C API to the Rust wtransport library, but haven't actually written any of the FFI yet as I hadn't found someone else doing similar yet. I thought it would be ideal if there were a native library with the same API as whatever happens in Emscripten.)

If you would like to get started on this, you could make a draft PR against Emscripten with some API proposal similar to websocket.h, but for WebTransport.

I'm a bit busy at the moment, but I'd potentially be interested in contributing. I think it would at least get the ball rolling. Two heads (or more) are better than one.

Any updates on this? I'm also highly interested in the applications for this, especially multiplayer games in the browser.

@opera-aberglund Sadly, not that I'm aware of.

If no one implements it, I will have to implement it myself eventually. Especially because there is no WebRTC support in Emscripten, so there are no UDP-like unreliable messaging options for web-targeted Rust / C / C++ games on Emscripten.

Hopefully, they will accept the pull request.

I'm just using WebSockets for now on the web target until my game is out of alpha state, and dealing with the network congestion/latency.

cc: @kripken @sbc100

WebTransport is essentially WebSocket 2.0 with many new features including better security. As it is not compatible with WebSockets a new name were chosen. The reason for this is that making WebSockets secure required breaking changes of the protocol which would mess up all existing code that depends on the WebSockets API so a completely new protocol were designed.

Assume that a few years from now browsers will ship with WebSockets disabled per default making WebTransport the only viable alternative. So even if you are fully satisficed with WebSockets functionality it will become a maintenence problem when browsers no longer support it.

Another important change with WebTransport is that it is part of HTTP/3 while WebSockets is distinct from HTTP except by using the same ports. This means that when you make a WebTransport connection it uses the very same QUIC connection that HTTP is already using and as such is a lot more firewall friendly. It also share TLS connection as that is handled by QUIC which means that things like session tracking becomes a lot easier.