conveyor
conveyor copied to clipboard
Add rate limiter to all sources where bulk load is used
ES has a queue limit for bulk operations so either I need to insert more rows/request or I need to put a rate limiter so that each request has a chance to finish before the next one starts. Ideally I'd add a loop of some sorts that only releases one more for each one finishing...
As it is right now inserts will fail if there are too many bulk requests.