Memory-friendly behaviour for sending and receiving data on WebSocket

[acarstoiu]

In the current implementation of the WebSocket protocol, the server can send or receive entire messages only, which is a pitty considering the consumption of memory caused by assembling those byte arrays in full just to deserialize or serialize JS objects from/into them. Multiply this by the number of concurrent connections, which should be in the thousands.

Also consider the case where people need to send fair amounts of data (like files) on the connection. Yes, there are other protocols designed for file transfer, but WebSocket is more versatile 😉

My proposal is to bring streaming behaviour to this package by adding API for sending and receiving messages by fragments:

1. A new sendByFragments method on WebSocket which parallels the existing send: sendByFragments(message: AsyncIterable<RecognizedString>, isBinary?: boolean, compress?: boolean) : Promise<number>, along with an enhancement to the current send to also accept an Iterable<RecognizedString> as message send(message: RecognizedString|Iterable<RecognizedString>, isBinary?: boolean, compress?: boolean) : number, for the case when the serialized data is already loaded in memory (needs not be fetched from some storage). 👉 Mind that some RecognizedStrings are Iterable, so the fragmented variant should be executed when the message parameter is not a RecognizedString. In both variants iteration should be suspended when the message fragment is dropped due to backpressure, by calling return() on the iterator (call it without any guard, so as to crash when return() isn't defined). This is in line with the behaviour of a ReadableStream's asynchronous iterator.
2. A new messageFragments property on WebSocketBehavior, mutually exclusive with message: messageFragments?: (ws: WebSocket<UserData>, isBinary: boolean) => ((fragment: ArrayBuffer, isLast: boolean) => void | Promise<void>)) It returns a handler for message fragments which µWebSockets.js can call for every received fragment of a message (with the first call expected immediately after this method returns).

The fragments of a WebSocket message need not reflect the boundaries of frames that make up that message (you can have continuation frames after a message frame). The relationship between fragmentation and WebSocket frames should be decided internally by your implementation, perhaps steered by a new configuration value specified in WebSocketBehavior.

[uNetworkingAB]

There are already fragmented send functions. I don't rmember their name but pretty sure there are 3 of them. For receiving, typically people don't receive lots of data over WebSockets. You have maxPayload for this. Either way; how do you expect to assemble a JSON string for parsing unless it is buffered until completion? JSON.parse requires the entire buffer, there is no SAX parsing for JSON in JS what I know of, at least not widely used. Point being: extremely few people care about this feature.

[acarstoiu]

I don't rmember their name but pretty sure there are 3 of them.

Could you please update the documentation, then? Thank you.

Either way; how do you expect to assemble a JSON string for parsing unless it is buffered until completion?

I don't use JSON for serialization. Not unlike you, I prefer using performant tools or building them if not available.

Point being: extremely few people care about this feature.

Well, your argument doesn't cover the sending part, plus I can assure you that everybody will care once WebSocket (or better WebSocket multiplexed over QUIC) becomes the protocol of choice for transfering large amounts of data. If not for other reasons, simply because most applications already use WebSocket connections.

[uNetworkingAB]

sendFirstFragment, sendFragment, sendLastFragmet are under WebSocket

[uNetworkingAB]

They are missing from documentation yes https://unetworking.github.io/uWebSockets.js/generated/interfaces/WebSocket.html

[uNetworkingAB]

Btw, fragmentation sounds like an obvious thing to expose to user code but in practice, rarely anyone cares. The first prototype of this library exposed fragments and everyone hated it and wanted full messages instead. And since most websocket messages are small (it's not common to send GB of files in a single websocket message), it's not a problem in practice

[uasan]

Many private video services send video streams via WebSocket, Chrome has a good client solution for working with this: https://developer.mozilla.org/en-US/docs/Web/API/WebSocketStream/WebSocketStream

[acarstoiu]

Turns out there's a problem with publish: it may interfere with the fragments of a message (which correspond to separate frames, thus the 3-method API). Could you buffer published messages (or even drop them if they start piling) until the next call to send with fin=true is finished, if an unmatched call to send with fin=false has begun?

Off topic: if some (small) message gets published, will a subscribing socket push the message to the connected client in the absence of any further communication on that socket? I'm asking this because apparently you are buffering notifications until the next send call on that socket (sorry if I'm wrong, I do not speak C++).

[acarstoiu]

Here's the patch for docs/index.d.ts (file attachment does not work):

--- a/docs/index.d.ts	2025-04-25 18:48:44.804922153 +0300
+++ b/docs/index.d.ts	2025-04-25 19:21:12.153940612 +0300
@@ -48,12 +48,38 @@
  * Read more about this in the user manual.
  */
 export interface WebSocket<UserData> {
-    /** Sends a message. Returns 1 for success, 2 for dropped due to backpressure limit, and 0 for built up backpressure that will drain over time. You can check backpressure before or after sending by calling getBufferedAmount().
+    /** Sends a message. In terms of sending data, a call to this method must not follow a call to WebSocket.sendFirstFragment or WebSocket.sendFragment.
+     *
+     * Returns 1 for success, 2 for dropped due to backpressure limit, and 0 for built up backpressure that will drain over time. You can check backpressure before or after sending by calling getBufferedAmount().
      *
      * Make sure you properly understand the concept of backpressure. Check the backpressure example file.
      */
     send(message: RecognizedString, isBinary?: boolean, compress?: boolean) : number;
 
+    /** Sends the first frame of a multi-frame message. More fragments are expected (possibly none), followed by a last fragment. Care must be taken so as not to interleave these fragments with whole messages (sent with WebSocket.send) or with other messages' fragments.
+     *
+     * Returns 1 for success, 2 for dropped due to backpressure limit, and 0 for built up backpressure that will drain over time. You can check backpressure before or after sending by calling getBufferedAmount().
+     *
+     * Make sure you properly understand the concept of backpressure. Check the backpressure example file.
+     */
+    sendFirstFragment(message: RecognizedString, isBinary?: boolean, compress?: boolean) : number;
+
+    /** Sends a continuation frame of a multi-frame message. More fragments are expected (possibly none), followed by a last fragment. In terms of sending data, a call to this method must follow a call to WebSocket.sendFirstFragment.
+     *
+     * Returns 1 for success, 2 for dropped due to backpressure limit, and 0 for built up backpressure that will drain over time. You can check backpressure before or after sending by calling getBufferedAmount().
+     *
+     * Make sure you properly understand the concept of backpressure. Check the backpressure example file.
+     */
+    sendFragment(message: RecognizedString, compress?: boolean) : number;
+
+    /** Sends the last continuation frame of a multi-frame message. In terms of sending data, a call to this method must follow a call to WebSocket.sendFirstFragment or WebSocket.sendFragment.
+     *
+     * Returns 1 for success, 2 for dropped due to backpressure limit, and 0 for built up backpressure that will drain over time. You can check backpressure before or after sending by calling getBufferedAmount().
+     *
+     * Make sure you properly understand the concept of backpressure. Check the backpressure example file.
+     */
+    sendLastFragment(message: RecognizedString, compress?: boolean) : number;
+
     /** Returns the bytes buffered in backpressure. This is similar to the bufferedAmount property in the browser counterpart.
      * Check backpressure example.
      */

Documentation must be regenerated, after changing its title in docs/tsconfig.json.

[uNetworkingAB]

If you want to use fragmented sends you cannot use regular sends in between, that includes being subscribed to something that publishes.