data-prepper
data-prepper copied to clipboard
Improve OpenSearch bulk retry logging
Is your feature request related to a problem? Please describe.
It's difficult to track bulk retry failures if Data Prepper is writting to multiple indexes.
Describe the solution you'd like
-
Log the bulk failure the first time that there is a failure instead of only doing it every 5 times.
-
Update logging to show the OpenSearch index for which the bulk request failed.
https://github.com/opensearch-project/data-prepper/blob/ed3f75ed3beedfb6d72af1f4b63ae0d085f72a0b/data-prepper-plugins/opensearch/src/main/java/org/opensearch/dataprepper/plugins/sink/opensearch/BulkRetryStrategy.java#L243-L244
Describe alternatives you've considered (Optional) N/A
Additional context N/A