parquet-java icon indicating copy to clipboard operation
parquet-java copied to clipboard

PARQUET-869 Configurable min/max record counts for block size check

Open pradeepg26 opened this issue 7 years ago • 4 comments

Make min/max record counts for block size check are no longer hard coded inside InternalParquetRecordWriter.

pradeepg26 avatar Jan 11 '18 02:01 pradeepg26

Thanks for doing this, @pradeepg26! We've occasionally had use cases where it would have been nice. My two main concerns are the unnecessary boolean controlling whether or not to estimate when to do the next check and the naming. Naming should use "row group" instead of "block".

rdblue avatar Jan 31 '18 16:01 rdblue

bump? note disclosure on https://eng.uber.com/petastorm/

pwais avatar Jul 04 '19 11:07 pwais

@pwais, this was replaced by #470 that includes updates for problems in this PR. Unfortunately, other committers decided they did not want to change internal APIs so it was not committed.

rdblue avatar Jul 04 '19 17:07 rdblue

Thank you!

pwais avatar Jul 04 '19 22:07 pwais