rerun-io
Stop re-encoding transport data in `re_grpc_server`
re_grpc_server is a pure proxy server, who's only job is to relay messages for late-connecting clients. But currently it wastes a lot of CPU time on encoding/decoding arrow-data into protobuff, including expensive IPC encode/decode and LZ4 compress/decompress. As far as I can tell, there is no reason for this.
Related
- https://github.com/rerun-io/rerun/issues/9812
- https://rerunio.slack.com/archives/C041NHU952S/p1749845022757869
We've talked about this a bit on Slack, and our conclusion is that if we want a smart proxy (and we do, at some point), then we need to at least decode the messages. BUT, we shouldn't need to re-encode them again.
We've talked about this a bit on Slack, and our conclusion is that if we want a smart proxy (and we do, at some point), then we need to at least decode the messages. BUT, we shouldn't need to re-encode them again.
I wasn't part of this discussion so not sure what's involved exactly in "smart proxy" here, but normally all data necessary for routing purposes is present at the transport/protobuf layer: there is never any need to decode application/arrow data during routing (Redap for example never decodes any Arrow).
Just to clarify this issue a bit:
- We never perform IPC encoding/decoding in the gRPC proxy itself. The proxy implements the message proxy service trait, and the messages received are stored directly as the protobuf structs (the "transport" layer). No actual IPC encoding/decoding happens at any point.
- We have a
spawn_with_recvfunction to spawn the server, and also receive any data sent to it. This is used when spawning the Viewer, so that the Viewer doesn't need to connect to the server in the same process, it instead talks to it via shared memory (the channel). In that case, we decode from transport to application in order for the Viewer to be able to ingest the data. This exists purely for convenience and removes some otherwise unnecessary copies when going through a loopback network interface.
So the issue isn't about encoding/decoding application-level messages, but it's more about not having to re-encode the transport-level messages once they've been received. Which we can't do because. That's how the tonic APIs work!
