parquet-dotnet icon indicating copy to clipboard operation
parquet-dotnet copied to clipboard

Published 20 hours ago • verified publisher icon elastacloud

Return total row group size in bytes

Open spektom opened this issue 4 years ago • 0 comments

Fixes

Issue #452

Description

This PR adds two properties to ParquetRowGroupWriter that return total written size in bytes. Without this information it's impossible to optimize for written block size based solely on row numbers count, because this number depends on the schema, compression, encoding, ordering, etc.

  • [V ] I have included unit tests validating this fix.
  • [V] I have updated markdown documentation where required.
  • [V] I understand that successful approval of my pull request requires reproducible tests as per Contribution Guideline.

spektom avatar Feb 27 '20 13:02 spektom