examples
examples copied to clipboard
Consider adding Storage Connectors
I see there are examples for JDBC and Elastic, so perhaps the storage based connectors can be included as well?
HDFS: A Hadoop installation shouldn't be required since HDFS URI can be set to local filesystem. I've tested with hdfs.url=file:///tmp/kafka-connect
, and it worked as expected.
S3: An AWS account isn't required. See - https://blog.minio.io/journaling-kafka-messages-with-s3-connector-and-minio-83651a51045d
I have some workable code for at least S3 here - https://github.com/cricket007/kafka-connect-sandbox
Extras: Use something like Apache Drill to connect to the local Zookeeper Server and run Drill SQL queries over data (can query JSON and Avro files)
@cricket007 : thanks for the suggestion, we'll look into it.
Minio example: https://github.com/framiere/a-kafka-story/tree/master/step13