csv2parquet
csv2parquet copied to clipboard
Very bad memory performance: csv2parquet requires tons of memory
If I read in a file that's a few GB big, it crashes my computer with 16GB because it swaps like crazy.
The whole input file should never be in memory as a whole but streamed out.