Benchmark.jl icon indicating copy to clipboard operation
Benchmark.jl copied to clipboard

Print % of Wins, and rename @benchmark for 2 exp

Open diegozea opened this issue 13 years ago • 0 comments

% of wins on runnings gives and idea of how consistent is the difference. It's useful now. I'm using it and avoids me some bad interpretations. In the future some statistical test can help even more. The two macros are crashing, I rename one to benchmark2 in order to avoid:

julia> @benchmark(quote f() end,quote g() end,"cmp","f","g",100)
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
f wins to g on the 8.791208791208792% of runnings
g wins to f on the 7.6923076923076925% of runnings
Ties: 83.51648351648352%

2x12 DataFrame:
        BenchmarkCategory BenchmarkName Iterations  TotalWall AverageWall    MaxWall MinWall             Timestamp            JuliaVersion    JuliaHash CodeHash      OS
[1,]                "cmp"           "f"         91 6.91414e-6  7.59795e-8 1.19209e-6     0.0 "2013-01-09 16:25:24" "0.0.0+106748552.r7d5d" "7d5debe4ef"       NA "Linux"
[2,]                "cmp"           "g"        109 8.82149e-6  8.09311e-8 1.19209e-6     0.0 "2013-01-09 16:25:24" "0.0.0+106748552.r7d5d" "7d5debe4ef"       NA "Linux"


julia> @benchmark(quote f() end,"cmp","f",100)
in @benchmark: wrong number of arguments

Note there are some push to rename in DataFrames Best

diegozea avatar Jan 09 '13 19:01 diegozea