BenchmarkTools.jl
BenchmarkTools.jl copied to clipboard
Document necessity to interpolate variables into top-level execution environment
julia> [@elapsed n for n in 1:1]
1-element Array{Float64,1}:
5.1e-8
julia> [@belapsed n for n in 1:1]
ERROR: UndefVarError: n not defined
Stacktrace:
[1] ##core#667() at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:316
[2] ##sample#668(::BenchmarkTools.Parameters) at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:322
[3] #_run#10(::Bool, ::String, ::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:350
[4] (::BenchmarkTools.#kw##_run)(::Array{Any,1}, ::BenchmarkTools.#_run, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at ./<missing>:0
[5] anonymous at ./<missing>:?
[6] #run_result#19(::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:44
[7] (::BenchmarkTools.#kw##run_result)(::Array{Any,1}, ::BenchmarkTools.#run_result, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at ./<missing>:0
[8] #run#21(::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:67
[9] (::Base.#kw##run)(::Array{Any,1}, ::Base.#run, ::BenchmarkTools.Benchmark{Symbol("##benchmark#666")}, ::BenchmarkTools.Parameters) at ./<missing>:0 (repeats 2 times)
[10] (::##8#9)(::Int64) at /Users/andreasnoack/.julia/v0.6/BenchmarkTools/src/execution.jl:234
[11] collect(::Base.Generator{UnitRange{Int64},##8#9}) at ./array.jl:470
Shouldn't that be done with interpolation?
julia> [@belapsed $n for n in 1:1]
1-element Array{Float64,1}:
3.8e-11
Benchmarking always occurs at top-level by design, this isn't a bug.
dup of https://github.com/JuliaCI/BenchmarkTools.jl/issues/109
see also https://github.com/JuliaCI/BenchmarkTools.jl/issues/65#issuecomment-313926098.
This really should be documented in more places (in the macro docstrings?) since people have run into it so many times.