Document profiling with @bprofile

[simonbyrne]

It would be fantastic if there was a way to get a profile trace of an @benchmarkable object, and exploit some of the nice features (i.e. run it enough times to get a good sample). Any ideas on ways to do this?

[simonbyrne]

At the moment, I can do something like:

julia> b = @benchmarkable sin(0.1)
BenchmarkTools.Benchmark{Symbol("##benchmark#277")}(BenchmarkTools.Parameters(5.0,10000,1,0,true,false,0.05,0.01))

julia> tune!(b)
BenchmarkTools.Benchmark{Symbol("##benchmark#277")}(BenchmarkTools.Parameters(5.0,10000,999,0,true,false,0.05,0.01))

julia> @profile run(b)
BenchmarkTools.Trial: 
  samples:          10000
  evals/sample:     999
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  0.00 bytes
  allocs estimate:  0
  minimum time:     9.00 ns (0.00% GC)
  median time:      9.00 ns (0.00% GC)
  mean time:        10.55 ns (0.00% GC)
  maximum time:     199.00 ns (0.00% GC)

julia> Profile.print()
137 ./event.jl:68; (::Base.REPL.##3#4{Base.REPL.REPLBackend})()
 137 ./REPL.jl:95; macro expansion
  137 ./REPL.jl:64; eval_user_input(::Any, ::Base.REPL.REPLBackend)
   137 ...urces/julia/lib/julia/sys.dylib:?; eval(::Module, ::Any)
    137 ./boot.jl:234; eval(::Module, ::Any)
     137 ./<missing>:?; anonymous
      137 ./profile.jl:16; macro expansion;
       137 ...nchmarkTools/src/execution.jl:27; run(::BenchmarkTools.Benchmark{Symbol("##benc...
        137 ...nchmarkTools/src/execution.jl:27; #run#17(::Array{Any,1}, ::Function, ::Benchm...
         137 ./<missing>:0; (::BenchmarkTools.#kw##_run)(::Array{Any,1},...
          62 ...chmarkTools/src/execution.jl:270; #_run#2(::Bool, ::String, ::Array{Any,1}, :...
          75 ...chmarkTools/src/execution.jl:276; #_run#2(::Bool, ::String, ::Array{Any,1}, :...
           1  ./int.jl:124; ##sample#279(::BenchmarkTools.Parameters)
           71 ...chmarkTools/src/execution.jl:248; ##sample#279(::BenchmarkTools.Parameters)
            64 ./<missing>:0; ##core#278()
             2  ./math.jl:196; sin(::Float64)
             55 ./math.jl:202; sin(::Float64)
              4 ./math.jl:196; nan_dom_err
           1  ...chmarkTools/src/execution.jl:251; ##sample#279(::BenchmarkTools.Parameters)

Basically, I want a way to get rid of all the upper elements of the trace, and display only:

            64 ./<missing>:0; ##core#278()
             2  ./math.jl:196; sin(::Float64)
             55 ./math.jl:202; sin(::Float64)
              4 ./math.jl:196; nan_dom_err

[jrevels]

This would indeed be useful (in the same vein, it might also be nice to provide an easy hook into https://github.com/carnaval/LinuxPerf.jl for Linux users).

For the display problem, I guess it'd be sufficient to simply do a find-and-delete of any parts of the trace that are in BenchmarkTools/src. The API for this would probably end up looking like BenchmarkTools.run(::BenchmarkTools.Benchmark, profile = true).