haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Limit rate of ingestion at haystack-collector for incoming spans

Open doctorXWrites opened this issue 6 years ago • 1 comments
trafficstars

This is to safeguard the system from getting overwhelmed in case of unplanned extreme traffic increase. This does not mean we are trading anything with scalability. This is just to ensure the smooth capacity planning and scalability for downstream services in case of onboarding new services and handle unplanned traffic increases if resources are constrained.

Guava rate limiter is one option that could be used here.

doctorXWrites avatar Sep 25 '19 07:09 doctorXWrites

Here is WIP branch https://github.com/ExpediaDotCom/haystack-collector/compare/rate_limiter

doctorXWrites avatar Sep 29 '19 22:09 doctorXWrites