feat(edda): impl a `CompressingStream` to reduce number of Edda requests
This change introduces a CompressingStream into the Edda server which wraps the underlying Edda requests subscription. It reads message from the upstream subscription, and attempts to "compress" multiple API requests into a single, new compressed request.
The simplified happy path for the stream is as follows:
-
Reading the first message from the subscription
The message is parsed to determine how many messages are pending behind this message. That pending number plus the first message determine the "read window" for the state machine. That is, how many messages to immediately consume before compressing.
-
Parsing the first message into an API request
The message is parsed into an API request using the
ApiTypesNegotiatefunctionality. -
Reading the next message from the subscription in the read window
The state is looped back upon until all messages in the read window are consumed.
-
Parsing the next message into an API request in the read window
After reading the message, it is parsed into an API request, then it returns to reading the next message (in 3) until all messages in the read window are read and parsed.
-
Compressing multiple API requests into a single compressed request
All the API requests are fed into the
CompressedRequest::from_requests()function, which reduces multiple API requests into oneCompressedRequesttype. This is where the "compression" takes place. -
Deleting a message from the Jetstream stream
This state is looped back upon until all Jetstream messages are deleted, as they have been successfully read, parsed, and compressed.
-
Converting request into final message to yield from [
Stream]The final
CompressedRequesttype is serialized into aLocalMessagewhich implements thenaxum::MessageHeadtrait, thus enabling this stream to be the incoming stream for a Naxum app.