fst icon indicating copy to clipboard operation
fst copied to clipboard

Add benchmark vs `arrow::read/write_parquet`

Open xiaodaigh opened this issue 6 years ago • 2 comments
trafficstars

The new arrow package is now online, so can add arrow::read/write_parquet/arrow. I don't mind contributing to this but I will need to finish off a couple of CRAN submissions.

xiaodaigh avatar Aug 26 '19 07:08 xiaodaigh

Hi @xiaodaigh, thanks for your request!

Indeed, we should create some extensive benchmarks to compare arrow and fst. Perhaps we could take the benchmark code from #177 (posted [here])(https://gist.github.com/fstpackage/e710cf2adaa02f96bc8dbee9318ba733) as a start as it compares the performance for various types, number of rows and threads.

Your contribution would be much appreciated!

MarcusKlik avatar Aug 27 '19 09:08 MarcusKlik

I've posted some initial benchmarks here and more detailed measurements are forthcoming in this repo.

MarcusKlik avatar Sep 03 '19 11:09 MarcusKlik