lighthouse
lighthouse copied to clipboard
Missing partitionBy for existing table in MultiplePartitionOverwrite mode.
Hi Mikalai,
the line you added causes the unit tests to fail. In case you can't access the logs from our CI tool:
sbt.ForkMain$ForkError: org.apache.spark.sql.AnalysisException: insertInto() can't be used together with partitionBy(). Partition columns have already been defined for the table. It is not necessary to use partitionBy().;
I'm not up to speed with what's going on here, but it seems to me that if a table already exists, it should not be possible to redefine the partitions.
What is the behaviour you're trying to allow?