Ryan Scott

In order to ensure that `criterion` generates enough data for us to have some trust in the raw measurements, `criterion` sets a minimum time threshold that a benchmark must run...

Some helpful information to know: * What exactly is the issue you are experiencing? The title mentions "negative measurements", but it isn't clear to me what that means. Are you...

I agree that we should offer more control over how this works for the benefit of power users. See also #210/#218, where users have requested the ability to reduce the...

The [`env`](https://hackage.haskell.org/package/criterion-1.5.6.2/docs/Criterion.html#v:env) function is usually recommended for inputs that take a long time to compute. Moreover, I'm not sure if using `deepseq` in `nf` will actually avoid the input from...

> What do you think about forcing the `Benchmarkable` in the place where it's going to be executed prior to execution? Forcing the `Benchmarkable` itself probably isn't going to do...

Hm. I'm not quite sure what to make of this, since the choice of `performMinorGC` was made deliberately in #187 after some careful investigation. I'm a bit reluctant to simply...

I think the issue might have been that the `FromJSON Measured` instance was doing something dodgy with `null` values behind the scenes, which we fixed with [this commit](https://github.com/bos/criterion/commit/bfa9dbe062f3b25112801c837c0c0bae854c5644). Therefore, I'm...

Ack, I spoke too soon. I can still reproduce this error with `criterion-1.1.4.0`.

I did some digging into this recently, and I think I've narrowed the issue down to a function in `statistics`. Notice [these lines](https://github.com/bos/criterion/blob/f117bec84c9bf460549fa565b8f30fac075a7db7/Criterion/Analysis.hs#L202-L203) in `criterion`: ``` haskell (coeffs,r2)

Here's what I believe to be a more deterministic way of reproducing the bug, using `initialize` instead of `createSystemRandom` (the latter of which is what `criterion` actually uses internally): ```...