lighthouse icon indicating copy to clipboard operation
lighthouse copied to clipboard

Missing partitionBy for existing table in MultiplePartitionOverwrite mode.

Open surtamikalai opened this issue 6 years ago • 1 comments

surtamikalai avatar Mar 21 '19 15:03 surtamikalai

Hi Mikalai,

the line you added causes the unit tests to fail. In case you can't access the logs from our CI tool:

sbt.ForkMain$ForkError: org.apache.spark.sql.AnalysisException: insertInto() can't be used together with partitionBy(). Partition columns have already been defined for the table. It is not necessary to use partitionBy().;

I'm not up to speed with what's going on here, but it seems to me that if a table already exists, it should not be possible to redefine the partitions.

What is the behaviour you're trying to allow?

oliverw1 avatar Mar 21 '19 18:03 oliverw1