Andy Grove
Andy Grove
@parthchandra I made some notes above on the `core3` failures. Do you know how we should handle missing columns in this case?
Failure count: ``` core-1: Failed: Total 9216, Failed 17, Errors 0, Passed 9199, Ignored 214, Canceled 6 core-2: Failed: Total 2655, Failed 19, Errors 0, Passed 2636, Ignored 387 hive-1:...
Thanks for the reviews @parthchandra and @kazuyukitanimura
If I disable Comet shuffle, then the problem seems to go away, so it does look like an issue specific to Comet columnar shuffle.
We should also start testing with all supported data types for the array expressions.
Here is an example of an issue that we should have been able to catch using fuzz testing: https://github.com/apache/datafusion-comet/issues/1307
> Would it help to extend makeParquetFileAllTypes or create a new method which creates a parquet file that contains more complex structures like lists (with all datatypes and structs because...
> Hi [@andygrove](https://github.com/andygrove) do you have a reproduce case, or at least a schema. `native-datafusion` reader should handle different kind of array/struct combintations This is the reason I was getting...
Fixed in https://github.com/apache/datafusion-comet/pull/1667
@radhikabajaj123 What happens if you run the following command from the terminal? ```shell ls -l /Users/radhika.bajaj/Documents/datafusion-comet/native/proto ``` The issues seems to be with permissions to files in that folder?