kafka-connect-hdfs icon indicating copy to clipboard operation
kafka-connect-hdfs copied to clipboard

duplicate column in hive table when use FieldPartitioner and partition.field.name

Open raybin7 opened this issue 4 years ago • 0 comments

I have table like this

id       name     year     month     day
emp01    karl     2021      05        01

and use FieldPartitioner, partition.filed.name is "year".

data commit succeeded, but when I use select command in presto, this failed like this Query _____ failed: Hive metadata for table ___ is invalid: Table descriptor contains duplicate colums.

So I searched about this error, found it https://github.com/confluentinc/kafka-connect-hdfs/issues/221

This error may be fixed, but it is not applied at Orc format.

How can I fix it? Should I use Avro format?

raybin7 avatar Jun 07 '21 06:06 raybin7