kafka-connect-storage-common
kafka-connect-storage-common copied to clipboard
Conflict between partition fields and the data source
We are using DailyPartitioner for our HDFS sink connector with hive integration. The source of a topic is from Debezium that captures the change from a source table that contains fields called month, year.
This leads to the situation where we have ambiguous fields (two year columns in hive table - one creates from the partitioner and another one from the source).
Is there any way to workaround this at the moment? Shouldn't we use the prefix for the generated partition fields to avoid the conflict? e.g. Use _year, _month, _day for DailyPartitioner