haskell-cpu-instruction-counter icon indicating copy to clipboard operation
haskell-cpu-instruction-counter copied to clipboard

Integrate with criterion?

Open jberryman opened this issue 7 years ago • 2 comments

It seems as though this both would complement and be improved by integration into criterion:

  • instruction counts as regression would be validated by criterion timing
  • a timing regression combined with no increase in instr count would be a really useful thing to know (e.g. it might point to a space leak)
  • standalone timing tests like the one in the example in the README here are fraught due to compiler optimizations; criterion is meant to solve this

What are your thoughts? I suppose it would need to be conditional on linux OS and be optional due to the permissions issues you mention

jberryman avatar Mar 17 '18 03:03 jberryman

  • it might provide a means of normalizing timing benchmarks, so that timing-based perf regression tests could be run on both, say, a powerful CI server and a devs local machine (I'm skeptical this would be effective, but it might be)

jberryman avatar Mar 17 '18 03:03 jberryman

I think it's a good idea, if criterion is happy to acquire a Linux-only feature that might run only in some environments.

nh2 avatar Jun 09 '18 14:06 nh2