bencheeorg
Option to give same scale to graphs
From twitter:
Would it be possible to set a scale of each chart the same by default ?
What I think this would/could entail is to set the scale of all raw tun time charts to be the same, probably also the same for the box plots. To do this effectively, it'd be convenient if benchee provided some meta statistics aka what are the minimum/maximum over the whole sample set (or well minimum of minimums/maximum of maximums).
Probably best implemented as an option, setting it as the default might also cause problems if the different jobs are wildly differing in how fast/slow they are.
Also, maybe @fazibear has some input as to what exactly he wishes for.
I think I should be the default. Why ? Because, when we read graphs, we want to compare them. When the scales will be the same you can see exact differences. I've made a mistake because of scales. Though that graphs are different, but with the same scale are almost the same (peaks changed the scale). It would be nice to have a button that will trim scale to the data of specific graph. But overall scales should be the same.
Thanks for the input!
You use case and mine vary a bit (which is fine) - i mostly use the raw run time charts and the histograms just to get a feel for the individual benchmarking job and its characteristics aka is anything getting slower, are there obvious spikes that might have ruined the results, is it getting faster at some point (do I need more warmup?) etc. are questions I most commonly ask myself when I look at them.
Still true, in a presentation I compared two and it would have been way better if they used the same scale.