pandas2 icon indicating copy to clipboard operation
pandas2 copied to clipboard

Published 20 hours ago • verified publisher icon wesm

DEV: Publishing benchmarks vs. trunk?

Open wesm opened this issue 8 years ago • 2 comments

We'll want to develop a microbenchmark suite to compare pandas 2.0 perf versus 1.x, especially in microbenchmarks. What's the right tool for this? asv, vbench?

wesm avatar Oct 15 '16 19:10 wesm

IMO certainly not vbench (unless someone wants to put time in developing, but even then). The benchmark suite already is ported over to asv, and I have a PR to clean them up (https://github.com/pandas-dev/pandas/pull/14099, once that is finished, I would remove the vbench ones, we don't use them anymore). But I have less of an idea how reliable it is for microbenchmarks (I think it is based on timeit, and I think there are arguments to be made that this is not really reliable for microbenchmarks)

jorisvandenbossche avatar Oct 15 '16 19:10 jorisvandenbossche

I was thinking that ASV is the best choice -- as long as we can easily compare perf vs. a snapshot of master (e.g. we can use v0.19.0 as a baseline) so we can see how we're doing.

wesm avatar Oct 16 '16 20:10 wesm