zeth icon indicating copy to clipboard operation
zeth copied to clipboard

Optimize data serialization and consider alternatives to protocol buffers

Open AntoineRondelet opened this issue 5 years ago • 2 comments
trafficstars

For now, we use protocol buffers along with gRPC for serializing/deserializing structured data in the RPC communications of the various components. We should:

  1. Consider using Arena Allocation (see:https://developers.google.com/protocol-buffers/docs/reference/arenas) to optimize memory allocation and performances, and
  2. Consider other serialization tools such as: Flatbuffers, as they offer appealing performances (see: https://google.github.io/flatbuffers/flatbuffers_benchmarks.html). These need to be considered at some point.

AntoineRondelet avatar Mar 17 '20 14:03 AntoineRondelet

See also: Cap'n Proto https://github.com/capnproto/capnproto and its python wrapper: https://github.com/capnproto/pycapnp

AntoineRondelet avatar Apr 08 '20 11:04 AntoineRondelet

See: https://github.com/clearmatics/zeth/pull/302#discussion_r529650654 Cap'n proto seems to support constants, which would make some of our shared configuration much easier to handle.

AntoineRondelet avatar Dec 15 '20 17:12 AntoineRondelet