criterion icon indicating copy to clipboard operation
criterion copied to clipboard

first benchmark is expensive

Open jwaldmann opened this issue 1 year ago • 5 comments

I expect identical output for the three benchmarks, but the first one (consistently) needs double time. (Possibly related: #60)

import Criterion.Main

main = defaultMain
  [ bench "1"   $ nf length [1 .. 10^7 :: Int ]
  , bench "2"   $ nf length [1 .. 10^7 :: Int ]
  , bench "3"   $ nf length [1 .. 10^7 :: Int ]
  ]


$ cabal run
benchmarking 1
time                 38.37 ms   (38.26 ms .. 38.50 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 38.67 ms   (38.53 ms .. 38.97 ms)
std dev              387.9 μs   (201.4 μs .. 535.6 μs)

benchmarking 2
time                 20.21 ms   (20.13 ms .. 20.31 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 20.10 ms   (20.03 ms .. 20.17 ms)
std dev              175.3 μs   (144.9 μs .. 234.5 μs)

benchmarking 3
time                 20.77 ms   (20.43 ms .. 21.09 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 20.24 ms   (20.14 ms .. 20.41 ms)
std dev              306.0 μs   (225.8 μs .. 397.1 μs)

For criterion-1.6.0.0, with ghc-8.10.7 .. ghc-9.6.1

jwaldmann avatar Mar 26 '23 15:03 jwaldmann