wormhole
wormhole copied to clipboard
Node: Observation batching
This PR introduces the batching of observations by adding the SignedObservationBatch gossip message. It modifies the guardian to wait up to one second in order to batch as many observations as possible (up to 4000) into a single gossip message. This not only reduces gossip traffic but also the load on the guardians and applications that listen to observations.
This PR is meant to go hand-in-hand with PR #4000, which further improves gossip traffic by splitting it across different topics, allowing the guardians and gossip listeners (like spies and flies) to only subscribe to the topics they need.
NOTE: Once the gossip network cutover is complete, support for SignedObservation (individual observation) messages will be dropped, but in the meantime, applications that listen to observations need to support both message streams.
A concern has been raised regarding possible latency impacts of this change. Therefore I am converting this PR back to draft until those concerns can be addressed.
The availability of Pyth's market data with as little latency as possible is a very strong usecase to warrant not being part of observation batching, and so they should be excluded from it.
This PR is being superseded by PR #4066, so this one is being closed.