kafka-connect-hdfs
kafka-connect-hdfs copied to clipboard
duplicate column in hive table when use FieldPartitioner and partition.field.name
I have table like this
id name year month day
emp01 karl 2021 05 01
and use FieldPartitioner, partition.filed.name is "year".
data commit succeeded, but when I use select command in presto, this failed like this Query _____ failed: Hive metadata for table ___ is invalid: Table descriptor contains duplicate colums.
So I searched about this error, found it https://github.com/confluentinc/kafka-connect-hdfs/issues/221
This error may be fixed, but it is not applied at Orc format.
How can I fix it? Should I use Avro format?