lodestar icon indicating copy to clipboard operation
lodestar copied to clipboard

Consider using snappy along with snappyjs

Open twoeths opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe.

Right now we only use snappyjs and it's good with current gossip message size, but after the merge the block size is way bigger and it's better to use "snappy" at least for beacon_block topic, it also support async version

  • The benchmark: https://github.com/ChainSafe/lodestar/blob/tuyen/benchmark-snappy/packages/lodestar/test/perf/network/gossip/encoding.test.ts#L33
  • The result
Uncompressed size Times of attestation size # of runs per second - snappyjs # of runs per second - snappy
229 bytes 1 815 120
22.9 kb 100 2022 2047
114 kb 500 3.6 8.6

Describe the solution you'd like

Consider using snappy for beacon_block topic and snappyjs for other topics

twoeths avatar Jun 17 '22 10:06 twoeths

another direction is to research if we can port the snappyjs implementation into assembly script since the interface is similar to as-sha256

twoeths avatar Jun 22 '22 08:06 twoeths

Reevaluate snappy locations based on current block sizes.

philknows avatar Nov 05 '23 15:11 philknows