kafka-connect-hdfs icon indicating copy to clipboard operation
kafka-connect-hdfs copied to clipboard

Published 20 hours ago • verified publisher icon confluentinc

Exactly-once should also work without topic name included in the path

Open hariprasad-k opened this issue 5 years ago • 2 comments

This change is necessary to retain EOS semantics to support flexible storage partitioning scheme as proposed here: https://github.com/confluentinc/kafka-connect-storage-common/issues/126

hariprasad-k avatar Sep 09 '20 13:09 hariprasad-k

@hariprasad-k this looks like something I am looking for too. What is holding the progress here? Did you manage to get this done somehow else?

JozoVilcek avatar Feb 04 '21 12:02 JozoVilcek

@hariprasad-k this looks like something I am looking for too. What is holding the progress here? Did you manage to get this done somehow else?

@JozoVilcek Yes, if you apply the below patches, it does work for us to exclude topic names with EoS semantics. We are still awaiting for committers to review this change proposal though, so that we don’t need to maintain this internal fork.

https://github.com/confluentinc/kafka-connect-storage-common/pull/127/files https://github.com/confluentinc/kafka-connect-hdfs/pull/528/files

hariprasad-k avatar Feb 04 '21 13:02 hariprasad-k