kafka-connect-hdfs icon indicating copy to clipboard operation
kafka-connect-hdfs copied to clipboard

Original file names instead of offset

Open AbdullahAlhulwah opened this issue 5 years ago • 1 comments

Is there a way to keep the original file name when storing in HDFS ? We are using kafka to move json files to HDFS, we want the names of the files stored in HDFS to be the same as the json files names, not the offset number.

AbdullahAlhulwah avatar Oct 25 '20 08:10 AbdullahAlhulwah

Currently, saving offset in the file name helps to create the exactly once semantics logic implementation

ghost avatar Jul 24 '21 06:07 ghost