hygiene in benchmark macros

[jsams]

Here's a minimal example (thanks to mbauman for helping identify the issue). I've run into other issues with @benchmark breaking on other example functions that I no longer have handy, which makes me think this is not limited to just the time name.

using BenchmarkTools
time = 2            
@benchmark identity(time)  
#=
ERROR: MethodError: objects of type Int64 are not callable
Stacktrace:
 [1] #_run#224(::Bool, ::String, ::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at /home/james/.julia/v0.6/BenchmarkTools/src/execution.jl:347
 [2] (::BenchmarkTools.#kw##_run)(::Array{Any,1}, ::BenchmarkTools.#_run, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at ./<missing>:0
 [3] anonymous at ./<missing>:?
 [4] #run_result#19(::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at /home/james/.julia/v0.6/BenchmarkTools/src/execution.jl:44
 [5] (::BenchmarkTools.#kw##run_result)(::Array{Any,1}, ::BenchmarkTools.#run_result, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at ./<missing>:0
 [6] #run#21(::Array{Any,1}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at /home/james/.julia/v0.6/BenchmarkTools/src/execution.jl:67
 [7] (::Base.#kw##run)(::Array{Any,1}, ::Base.#run, ::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}, ::BenchmarkTools.Parameters) at ./<missing>:0
 [8] warmup(::BenchmarkTools.Benchmark{Symbol("##benchmark#674")}) at /home/james/.julia/v0.6/BenchmarkTools/src/execution.jl:100
=#

[jrevels]

I guess calls e.g. https://github.com/JuliaCI/BenchmarkTools.jl/blob/00cc39e3e7043e0c9146921672f36e29e7c498e2/src/execution.jl#L347 need to be fully qualified

[jrevels]

there are likely still issues here if people want to tackle them

[cstjean]

It's basically the same issue, but I'll point out that defining time breaks any subsequent use of @btime:

julia> using BenchmarkTools

julia> time = 24
24

julia> @btime 1+1
ERROR: MethodError: objects of type Int64 are not callable
Stacktrace:
 [1] #_run#3(::Bool, ::String, ::Base.Iterators.Pairs{Symbol,Integer,NTuple{4,Symbol},NamedTuple{(:samples, :evals, :gctrial, :gcsample),Tuple{Int64,Int64,Bool,Bool}}}, ::Function, ::BenchmarkTools.Benchmark{Symbol("##benchmark#366")}, ::BenchmarkTools.Parameters) at /home/cst-jean/.julia/packages/BenchmarkTools/dtwnm/src/execution.jl:324

[ararslan]

Looks like the package is interpolating the wrong thing: Currently when calling functions inside of quoted blocks in macros, it's doing $BenchmarkTools.f(x), but it should be doing $(BenchmarkTools.f)(x). Note that doing this resolves the time binding in Main as Base.time, which means after calling @btime you can no longer define a variable called time.