parquet-java
parquet-java copied to clipboard
Reduce memory footprint for nulls caching
Scenario: There is a schema with many optional groups, e.g.
message example {
required binary id (UTF8);
optional group a1 (LIST) {
repeated int64 array;
}
optional group a2 (LIST) {
repeated int64 array;
}
...
optional group aN (LIST) {
repeated int64 array;
}
}
Many records without optional parameters are written. In this case groupNullCache will contain many elements which all are zeros.
Reporter: Boris Molodenkov
Related issues:
Note: This issue was originally created as PARQUET-1119. Please see the migration documentation for further details.