csv2parquet icon indicating copy to clipboard operation
csv2parquet copied to clipboard

Very bad memory performance: csv2parquet requires tons of memory

Open corneliusroemer opened this issue 2 years ago • 0 comments

If I read in a file that's a few GB big, it crashes my computer with 16GB because it swaps like crazy.

The whole input file should never be in memory as a whole but streamed out.

corneliusroemer avatar Dec 18 '21 02:12 corneliusroemer