Iterators.jl icon indicating copy to clipboard operation
Iterators.jl copied to clipboard

Run benchmarks on CI

Open iamed2 opened this issue 9 years ago • 2 comments

We should be able to run benchmarks on CI infrastructure and report on regressions or progressions. We may need to widen tolerance to account for variance in the cloud, but Curtis Vogt (@omus) has determined that it's feasible to rely on Travis-run benchmarks most of the time.

iamed2 avatar Sep 26 '16 18:09 iamed2

Just going to note that benchmarks on Travis CI do give a "ballpark" view for performance regressions. Definitely not great at finding subtle regressions.

omus avatar Sep 27 '16 14:09 omus

Mhm, they'll need to be run more seriously in a controlled (local) environment when examining.

iamed2 avatar Sep 27 '16 14:09 iamed2