David Venable

[BUG] Kafka Connect extension loaded even if not needed

1

**Describe the bug** When running a Data Prepper pipeline without any Kafka Connect plugins, I see logs for the Kafka extension. ``` 2023-10-27T14:44:40.828 [main] INFO org.opensearch.dataprepper.plugins.kafka.extension.KafkaClusterConfigExtension - Applying Kafka Cluster...

Support Kafka as a Sink

4

**Is your feature request related to a problem? Please describe.** Some pipeline authors want to send events from Data Prepper into Kafka. This can allow them to use an existing...

Consistent metric naming convention

1

### Background Data Prepper has an existing metric naming convention of: ``` {pipelineName}.{pluginType}.{metricName} ``` The `{metricName}` part is custom to each plugin. The `{pipelineName}.{pluginType}` is determined by the `PluginMetrics` class...

Pipeline Component Ids

3

# Introduction A Data Prepper pipeline can contain multiple sources, processors, and sinks with the same type. Presently, these cannot be distinguished. # Proposed Solution Data Prepper should assign a...

### Description This PR continues some of the work from #2378 and #4110 by updating the codecs to use the `EventFactory`. ### Issues Resolved N/A ### Check List - [...

### Description I created this script for some recent release notes for Data Prepper. It doesn't produce final results, but it has helped me get started. We could use this...

**Is your feature request related to a problem? Please describe.** The GeoIP database manifest contains a `sha256_hash` value. Data Prepper should verify this against the data downloaded to enhance security....

[RFC] OpenSearch Ingestion: A New Name for the Next Steps of Data Prepper

6

A lot has changed since Data Prepper was initially introduced to the public in December 2020. First, when Data Prepper launched there was no OpenSearch project. Shortly after the release...

Reduce the Data Prepper tar.gz and Docker image sizes

6

**Is your feature request related to a problem? Please describe.** It appears that the Data Prepper image has grown quite large. ``` ls -l release/archives/linux/build/distributions/ | awk '{print $5,$9}' 664652152...

**Is your feature request related to a problem? Please describe.** The `kafka` buffer and `kafka` source share a significant number of classes. Many of these classes have their own loggers....