BenchmarkTools.jl
BenchmarkTools.jl copied to clipboard
JuliaCI
A benchmarking framework for the Julia language
Here's a minimal example (thanks to mbauman for helping identify the issue). I've run into other issues with `@benchmark` breaking on other example functions that I no longer have handy,...
The following works fine ``` julia A = Array(SymTridiagonal(fill(2, 5), ones(5))) b = @benchmarkable Base.LinAlg.chol!(x) setup=(x = Hermitian(copy($A), :U)) warmup(b) run(b) ``` but injecting `tune!` between `warmup` and `run` as...
As a benchmark suite becomes large the likelihood of accidentally overwriting a previous group increases. It would be good if this would warn or error since it is very likely...
On Slack today, some timings were posted: ```jl julia> @btime sum(a) > 0 26.323 μs (1 allocation: 16 bytes) false julia> @btime any(a .> 0) 116.147 μs (5 allocations: 16.64...
It is not clear to me whether the output is based on the wall clock or CPU time. Could we add this to the documentation?
```julia julia> [@elapsed n for n in 1:1] 1-element Array{Float64,1}: 5.1e-8 julia> [@belapsed n for n in 1:1] ERROR: UndefVarError: n not defined Stacktrace: [1] ##core#667() at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:316 [2] ##sample#668(::BenchmarkTools.Parameters)...
`showall` is gone: https://github.com/JuliaLang/julia/pull/22847 How to best look at results now? Note that `repr` and `print` show even less info.
The current default timeout of five seconds is annoyingly large for simple benchmarking (if I want to benchmark four different versions of a simple loop, I have to wait 20s)....
Add simple stopwatch, see below for basic suggested implementation: ``` mutable struct StopWatch t1::Float64 t2::Float64 StopWatch() = new(NaN, NaN) end function start!(sw::StopWatch)::Nothing sw.t1 = time() Nothing() end function stop!(sw::StopWatch)::Nothing sw.t2...
Using something like https://github.com/bicycle1885/CodecZlib.jl the output files can be compressed to ~5% of the current size. It is a pretty lightweight dependency, so perhaps worth thinking about.