ericphanson
Add `comparison` function
Based on #13.
This is a draft because it needs more work, but I figured I'd put up what I have. Given two BenchmarkHistogram objects (the result of @benchmark), it can plot the histograms one after the other using the same bins:
julia> using BenchmarkHistograms
julia> b1 = @benchmark sin(x) setup=(x = rand())
samples: 10000; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
ns
(7.12 - 7.34 ] ██████████████████████████████▏6979
(7.34 - 7.56 ] ██▌581
(7.56 - 7.78 ] ▋124
(7.78 - 7.99 ] ▎57
(7.99 - 8.21 ] ▏6
(8.21 - 8.43 ] ▏2
(8.43 - 8.65 ] ▏1
(8.65 - 8.86 ] ▏4
(8.86 - 9.08 ] ▏9
(9.08 - 9.3 ] ▏11
(9.3 - 9.52 ] ████████▌1969
(9.52 - 9.73 ] ▍84
(9.73 - 9.95 ] ▍75
(9.95 - 10.17] ▎32
(10.17 - 20.58] ▍66
Counts
min: 7.125 ns (0.00% GC); mean: 7.724 ns (0.00% GC); median: 7.209 ns (0.00% GC); max: 20.583 ns (0.00% GC).
julia> b2 = @benchmark sin(x) setup=(x = 10rand())
samples: 10000; evals/sample: 999; memory estimate: 0 bytes; allocs estimate: 0
ns
(7.76 - 8.22 ] ████▎765
(8.22 - 8.68 ] ▏4
(8.68 - 9.14 ] ▏1
(9.14 - 9.6 ] ████████▎1521
(9.6 - 10.07] ██████████████████████████████▏5534
(10.07 - 10.53] ███████████▏2050
(10.53 - 10.99] ▎38
(10.99 - 11.45] ▏12
(11.45 - 11.91] ▏3
(11.91 - 12.38] ▏2
(12.38 - 12.84] ▏2
(12.84 - 13.3 ] ▏5
(13.3 - 13.76] ▏20
(13.76 - 14.22] ▎29
(14.22 - 16.31] ▏14
Counts
min: 7.758 ns (0.00% GC); mean: 9.796 ns (0.00% GC); median: 10.010 ns (0.00% GC); max: 16.308 ns (0.00% GC).
julia> comparison(b1, b2)
(7.12 - 7.62 ] ██████████████████████████████7578
(7.62 - 8.12 ] ▊167
(8.12 - 8.62 ] ▏4
(8.62 - 9.12 ] ▏14
(9.12 - 9.62 ] ████████1992
(9.62 - 10.11] ▊171
(10.11 - 10.61] ▏13
(10.61 - 11.11] ▏2
(11.11 - 11.61] ▏1
(11.61 - 12.11] 0
(12.11 - 12.6 ] 0
(12.6 - 13.1 ] 0
(13.1 - 13.6 ] 0
(13.6 - 14.1 ] 0
(14.1 - 20.58] ▎58
(7.12 - 7.62 ] 0
(7.62 - 8.12 ] ████767
(8.12 - 8.62 ] ▏6
(8.62 - 9.12 ] ▏1
(9.12 - 9.62 ] ███████▉1521
(9.62 - 10.11] ██████████████████████████████ 5837
(10.11 - 10.61] █████████▏1753
(10.61 - 11.11] ▎35
(11.11 - 11.61] ▏11
(11.61 - 12.11] ▏3
(12.11 - 12.6 ] ▏1
(12.6 - 13.1 ] ▏2
(13.1 - 13.6 ] ▏16
(13.6 - 14.1 ] ▎30
(14.1 - 20.58] ▏17
Counts
I think it should at least print a comparison of the summary stats as well, but also perhaps should plot them next to each other (reusing the bin labels) rather than consecutively. Of course, consecutive is a lot easier here since it's just more println's.
Codecov Report
Merging #14 (075e407) into eph/outliers (9d70bbf) will decrease coverage by
13.51%. The diff coverage is53.12%.
@@ Coverage Diff @@
## eph/outliers #14 +/- ##
=================================================
- Coverage 100.00% 86.48% -13.52%
=================================================
Files 3 4 +1
Lines 84 111 +27
=================================================
+ Hits 84 96 +12
- Misses 0 15 +15
| Impacted Files | Coverage Δ | |
|---|---|---|
| src/BenchmarkHistograms.jl | 100.00% <ø> (ø) |
|
| src/comparison.jl | 0.00% <0.00%> (ø) |
|
| src/simple_unicode_histogram.jl | 100.00% <100.00%> (ø) |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 9d70bbf...075e407. Read the comment docs.
I'd expect the comparison (maybe better compare??) to do left-right alignment, e.g. (sorry, hand modified, no code to back this idea up:)
julia> comparison(b1, b2)
0 (7.12 - 7.62 ] ██████████████████████████████7578
767████ (7.62 - 8.12 ] ▊167
6 ▏(8.12 - 8.62 ] ▏4
1 ▏(8.62 - 9.12 ] ▏14
1521 ███████▉ (9.12 - 9.62 ] ████████1992
5837 ██████████████████████████████ (9.62 - 10.11] ▊171
1753 █████████▏(10.11 - 10.61] ▏13
35 ▎(10.61 - 11.11] ▏2
11 ▏(11.11 - 11.61] ▏1
3 ▏(11.61 - 12.11] 0
1 ▏(12.11 - 12.6 ] 0
2 ▏(12.6 - 13.1 ] 0
16 ▏(13.1 - 13.6 ] 0
30 ▎(13.6 - 14.1 ] 0
17 ▏(14.1 - 20.58] ▎58
