connect
connect copied to clipboard
Elasticsearch output backoff should honor HTTP code `429`
hi!
the elasticsearch output has an exponential backoff parameter, which is great!
sadly it only retries on HTTP codes 5xx (function shouldRetry).
Elasticsearch expects its client to also backoff with specific code TOO_MANY_REQUESTS (429) (doc).
this error code is somewhat nominal in the scenario of sudden message bursts and clients such as Logstash handles this gracefully (they refer to this as a "stalling output" scenario).
there could be 2 possible way to handle this:
- add
429as a condition in functionshouldRetry - make the HTTP codes configurable, like the
backoff_onparameter of thehttpprocessor
if that would help you, i'd be more than happy to submit a PR with the chosen fix.
finally, thanks for building this solution. it's very well thought off.
cheers