haystack
haystack copied to clipboard
Limit rate of ingestion at haystack-collector for incoming spans
trafficstars
This is to safeguard the system from getting overwhelmed in case of unplanned extreme traffic increase. This does not mean we are trading anything with scalability. This is just to ensure the smooth capacity planning and scalability for downstream services in case of onboarding new services and handle unplanned traffic increases if resources are constrained.
Guava rate limiter is one option that could be used here.
Here is WIP branch https://github.com/ExpediaDotCom/haystack-collector/compare/rate_limiter