zeth
zeth copied to clipboard
Optimize data serialization and consider alternatives to protocol buffers
trafficstars
For now, we use protocol buffers along with gRPC for serializing/deserializing structured data in the RPC communications of the various components. We should:
- Consider using Arena Allocation (see:https://developers.google.com/protocol-buffers/docs/reference/arenas) to optimize memory allocation and performances, and
- Consider other serialization tools such as: Flatbuffers, as they offer appealing performances (see: https://google.github.io/flatbuffers/flatbuffers_benchmarks.html). These need to be considered at some point.
See also: Cap'n Proto https://github.com/capnproto/capnproto and its python wrapper: https://github.com/capnproto/pycapnp
See: https://github.com/clearmatics/zeth/pull/302#discussion_r529650654 Cap'n proto seems to support constants, which would make some of our shared configuration much easier to handle.