incubator-gluten
incubator-gluten copied to clipboard
apache
[GLUTEN-7675][VL] Support parquet write with complex data type(eg. MAP, ARRYY and STRUCT)
What changes were proposed in this pull request?
The velox has supported parquet writer with the complex data type, such as MAP, ARRYY and STRUCT , so we may support them in gluten.
(Fixes: https://github.com/apache/incubator-gluten/issues/7675)
How was this patch tested?
Add unittests
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Do we only need to enable Spark35 UT?
@JkSelf
@FelixYBW The existing unit tests in spark 32, 33, 34 ,35 can verify this change. @weixiuli The unit tests failed. Can you help to fix? Thanks.
