data-prepper
data-prepper copied to clipboard
Dynamic key_path_prefix in S3 DLQ
Is your feature request related to a problem? Please describe.
As a user of the S3 DLQ, I would like to dynamically determine the prefix to send failed documents to with Data Prepper expressions, similar to how the S3 sink path_prefix
can be dynamic
Describe the solution you'd like
sink:
- opensearch:
s3:
dlq:
key_path_prefix: "my-${/key}-prefix/"
Alternative Solution An alternative that would be more useful would be to create a pipeline DLQ that can send to the s3 sink, which supports dynamic path prefix