Gang Wu
Gang Wu
Yes, that would help a lot.
@Xuanwo Thanks for raising the discussion! It would be a good opportunity to grow the Apache ORC community.
If we need to regard the code as third-party and remove ASF license header, some refactoring work is required to isolate code from Intel QPL and then add the copyright....
@wpleonardo Sorry to hear that
Could you please check if an email sent from [email protected] is accidently moved to the spam folder? @cxzl25
@LuciferYang Could you please check the test case? It seems to be writing `def_level=1` to a column with `max_def_level=0`.
Yes I know that. The exception is thrown when building size statistics, which is a new feature and has caught similar issues in the test cases of parquet-mr. So I'd...
These lines are suspicious: https://github.com/apache/spark/blob/05c87e51a5e50d1c156211848693b66937f12a8f/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorizedSuite.scala#L501-L505 If `inputValues` do not have any null, `maxDef` is set to 0. However, `definitionLevels` for non-null value is set to 1, which exactly violates the...
Yes, that change looks reasonable. Thanks for verification! @LuciferYang
(I have to admit that it is a little bit aggressive to enable a new feature by default on the parquet side, sigh)