kafka-connect-storage-common
kafka-connect-storage-common copied to clipboard
Interface Partitioner is inefficient
The Partitioner interface design is inefficient. The generatePartitionedPath takes a topic, which is immutable per task, plus an encodedPartition, which is per-record. That leads to things such as confluentinc/kafka-connect-hdfs#224, in which generatePartitionedPath is ignored and a separate method is called that reproduces the behavior of DefaultPartitioner's implementation, minus the partition information.
Instead, a base method should take only topic, and encodedPartition should take that static result and the SinkRecord.