David Venable

## Summary The `kafka` buffer should support rotating the `encryption_key`. This is a high-level issue for supporting key rotation. Data Prepper is not doing any key rotation on KMS. Instead,...

Enhanced support for lists, maps, strings

1

**Is your feature request related to a problem? Please describe.** We have several related requests for transforming maps and lists. We also have the `list_to_map` and `map_to_list` processors. Can we...

The integration tests for Kafka running 2.8.1 fail consistently on GitHub. "Kafka plugin integration tests / integration-tests (11, 2.8.1) (pull_request_target) " Example run: https://github.com/opensearch-project/data-prepper/actions/runs/7976394878/job/21776875103?pr=4161

Provide an OpenSearch API source

2

**Is your feature request related to a problem? Please describe.** I'd like to be able to point existing OpenSearch clients that perform ingestion to Data Prepper instead. **Describe the solution...

Support object checksums in the S3 sink

1

**Is your feature request related to a problem? Please describe.** S3 supports object checksums to verify the content sent to S3. With this, S3 will reject files that do not...

**Is your feature request related to a problem? Please describe.** Initially, I'd like to be able to test the circuit breaker behavior using manual controls. Additionally, there may be value...

**Is your feature request related to a problem? Please describe.** Data Prepper has sources which can pull binary data (mostly in base64) format. And we are adding some new processors...

[BUG] Kafka buffer does not retain metadata, including the receipt timestamp

1

**Describe the bug** The `kafka` buffer does not include metadata. Thus, this information is lost when sent across the Kafka topic. **Expected behavior** All metadata from the source should be...

**Is your feature request related to a problem? Please describe.** The `s3` source supports S3's [bucket ownership verification](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-owner-condition.html) to protect against reading from buckets in unexpected accounts. The `s3` sink...

**Is your feature request related to a problem? Please describe.** Data Prepper supports CloudWatch as a metric destination for its own metrics. The data-prepper-core project includes a [cloudwatch.properties](https://github.com/opensearch-project/data-prepper/blob/eb94f4abbd4f83ce3b3ebe4d4478af7531f9bc36/data-prepper-core/src/main/resources/cloudwatch.properties) file. It...