PkgBenchmark.jl
PkgBenchmark.jl copied to clipboard
JuliaCI
Easy benchmark tracking for packages
I'm trying to use PkgBenchmark with one of the private packages we've developed at my company. We have a private registry. The registry and packages are stored on a self-hosted...
I don't know if I overlooked this in the docs, but I'd like to have a feature to only run certain suites when benchmarking a package. Benchmarking all suites in...
Could benchmarkpkg have an option to *not* tune the parameters before test? I tend to set them manuelly when I'm creating the benchmark ie.: ```julia @benchmarkable foo(1) evals=300 ``` Any...
I have different `seconds = X` additions to `@benchmarkable` i.e `@benchmarkable solve_x(1) seconds=10` and `@benchmarkable solve_y(2) seconds=2` but this doesn't seem to be considered when computing the ETA and the...
It would be great if Project.toml/Manifest.toml in `benchmark/` folder is instantiated so that it's easy to configure benchmarks in a CI. is it just a matter of making https://github.com/JuliaCI/PkgBenchmark.jl/blob/master/src/runbenchmark.jl#L244 this...
Many packages would benefit from using PkgBenchmark.jl to catch performance regressions, and adoption would increase if the barrier for usage was lowered even further. Wondering if any of the following...
Sometimes, it would be nice to be able to plot the results of a benchmark suite, or do some kind of numerical analysis. This would be easy if both `BenchmarkResults`...
Considering that `PkgBenchmark` seems to automatically cache tuning data, I was quite unsure under what circumstances (if ever) one would need to pass `retune=true` to the `benchmarkpgk` function. It would...
See https://juliaci.github.io/PkgBenchmark.jl/stable/define_benchmarks/.
In our benchmark suite, @gdalle and I had several tests named similarly to named-tuples: ```Julia N = 2 K = 3 SUITE["Foo"]["(N=$N, K=$K)"] = @benchmarkable foo($x) ``` In the table...
