weddell icon indicating copy to clipboard operation
weddell copied to clipboard

Implement Batching configuration for publishing

Open medikent opened this issue 4 years ago • 0 comments

The default batching behavior indicates a publishing delay for small message volumes:

The cost of batching is latency for individual messages, which are queued in memory until their corresponding batch is filled and ready to be sent over the network. To minimize latency, batching should be turned off.

This latency is problematic for point-to-point messaging in event driven architectures or similar use cases. We should add a batching configuration implementation to enable adjusting batching configuration.

See Google's PubSub Batching docs

PubSub was designed for high throughput and high message volumes and latency under low load has been experienced by many. See thisStackOverflow post and this Issue in the NodeJS pubsub library.

medikent avatar Oct 16 '20 17:10 medikent