Benchmark.jl
Benchmark.jl copied to clipboard
Print % of Wins, and rename @benchmark for 2 exp
% of wins on runnings gives and idea of how consistent is the difference. It's useful now. I'm using it and avoids me some bad interpretations. In the future some statistical test can help even more. The two macros are crashing, I rename one to benchmark2 in order to avoid:
julia> @benchmark(quote f() end,quote g() end,"cmp","f","g",100)
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
WARNING: push is deprecated, use push! instead.
f wins to g on the 8.791208791208792% of runnings
g wins to f on the 7.6923076923076925% of runnings
Ties: 83.51648351648352%
2x12 DataFrame:
BenchmarkCategory BenchmarkName Iterations TotalWall AverageWall MaxWall MinWall Timestamp JuliaVersion JuliaHash CodeHash OS
[1,] "cmp" "f" 91 6.91414e-6 7.59795e-8 1.19209e-6 0.0 "2013-01-09 16:25:24" "0.0.0+106748552.r7d5d" "7d5debe4ef" NA "Linux"
[2,] "cmp" "g" 109 8.82149e-6 8.09311e-8 1.19209e-6 0.0 "2013-01-09 16:25:24" "0.0.0+106748552.r7d5d" "7d5debe4ef" NA "Linux"
julia> @benchmark(quote f() end,"cmp","f",100)
in @benchmark: wrong number of arguments
Note there are some push to rename in DataFrames Best