johnmyleswhite
Register/Tag
@johnmyleswhite, this repo has been getting some attention recently, including a number of uses in Julia Issues. It seems quite good, and is quite useful. Are you up for registering it (and maybe moving it JuliaLang)?
It would be nice if https://github.com/jrevels/Benchmarks.jl could be merged into this repo before that (and maybe moved to JuliaLang since it is now more or less officially used as the backend for Base benchmarks). I don't know @johnmyleswhite's plans though and the decision is of course up to him.
@johnmyleswhite I've been using this extensively, and would like to see this registered. Happy to maintain this going forward as well.
I'd like to see what @jrevels thinks about merging his changes into this repo (or the other way around) and then getting this registered.
On my end, I had a lot of draft work on removing the problematic standard error computation. I will try to get that put into a PR at some point in case anyone wants to follow up on it. I'm unlikely to have time to do much Julia work until July.
I can organize some noncontroversial changes (e.g. tests) from my fork into digestable PRs next week, if you would like.
I should mention that over the past month or so, my work in this area has led me to an execution strategy that's fairly different from (but definitely inspired by) the Benchmarks.jl approach - mainly while investigating how to make the benchmarking process more robust to noise. I'm still fine-tuning the implementation, but it's once it's ready, it'll replace Benchmarks.jl + BenchmarkTrackers.jl as the backend for BaseBenchmarks.jl and our CI process.
I don't want to hold up progress here, though. We can figure out how to manage the competing implementations once my work stabilizes (and I write some documentation to explain it, of course).
