pyperformance icon indicating copy to clipboard operation
pyperformance copied to clipboard

Include an overall summary number in compare output

Open itamaro opened this issue 3 years ago • 2 comments

People often quote a single perf number when comparing versions (e.g. "Python 3.11 is 25% faster than Python 3.10")

The pyperformance compare command prints out the "change factor" per benchmark. It's not clear how to go from this list of numbers to a single overall number. Having the compare command compute and print it out would also help with consistency when quoting this single perf number (is it mean? geometric mean? only including significant results or all of them?)

itamaro avatar May 19 '22 03:05 itamaro

ok, just learned this can be done with pyperf

pyperf compare_to baseline.json change.json

itamaro avatar May 19 '22 03:05 itamaro

It would be worth updating pyperformance though.

ericsnowcurrently avatar May 24 '22 15:05 ericsnowcurrently