microbench icon indicating copy to clipboard operation
microbench copied to clipboard

Wrong R^2 values

Open dtolnay opened this issue 8 years ago • 0 comments

The R^2 values from this crate look unrealistically high - way higher than I would expect based on my experience with variance in benchmarks.

I skimmed through lib.rs and didn't see "log" or "ln" so I think that may be the issue - linear regression assumes homoscedasticity and that assumption is violated by using a geometrically distributed number of iterations.

  • https://en.wikipedia.org/wiki/Linear_regression#Assumptions
  • https://en.wikipedia.org/wiki/Homoscedasticity

You should be able to fix it by using log-log regression - take the log of both iterations and elapsed time before computing the linear regression.

dtolnay avatar Jan 29 '17 20:01 dtolnay