jwaldmann

measure it.

yeah, also https://github.com/jwaldmann/containers/blob/intmap-fromList/containers-tests/benchmarks/IntMap.hs

"Why don't you.." - because it's a drastic change that should be discussed first? Current benchmark: ``` defaultMain [ bench "lookup" $ whnf (lookup keys) m ... ``` my proposal...

Great analysis - but my conclusion is not > Commit https://github.com/haskell/containers/commit/e6fbb98a71c5a72e0805285cbbb799c5f978aceb causes GHCs performance tests to fail. but this instead: > GHC's (quite unpredictable) inlining mechanics causes GHCs performance tests...

> tested with multiple sizes OK, I'll look into this. Just for calibration, this (on my local machine) ``` time cabal test all --test-show-details=direct --test-options="--quickcheck-tests 1000 --quickcheck-max-size 100" real 0m20.246s...

now the test run takes 25 min in CI (for each ghc version). that's too much?

no, I applied globally. https://github.com/jwaldmann/containers/commit/3a3929dff0b32707e226adb6cb17efdeb644e698 In theory, your method (lots of small, some large, etc.) should be fine for all tests?

I did restrict the test sets that get larger parameters. It's still expensive, and the programming gets ugly. I am not an expert at tasty's not-quite-regexps, and CI's not-quite-bash. OTOH,...

The benchmark code size is small. The benchmark compile time can be made small (if dependecies are stripped down). The benchmark execution time can be parameterized (from a few seconds...

I removed extra dependencies, now this is a nice small benchmark: https://gitlab.imn.htwk-leipzig.de/waldmann/containers-benchmark