benchee icon indicating copy to clipboard operation
benchee copied to clipboard

Option to benchmark until confidence value is reached

Open PragTob opened this issue 9 years ago • 1 comments

I'm not sure how this works, but there are measures in statistics to be confident of your results (standard deviation we have and I think it is one of them) - with this given we could add an option that says benchmark until this confidence level is reached, with some timeout though so it doesn't run forever if results vary too much naturally.

PragTob avatar Jun 20 '16 19:06 PragTob

https://github.com/jlouis/eministat seems to implement a confidence interval in Erlang + even outlier analysis. Potential reuse and inspiration over there - thanks/via @michalmuskala

PragTob avatar Dec 09 '17 14:12 PragTob