microbench
microbench copied to clipboard
Wrong R^2 values
The R^2 values from this crate look unrealistically high - way higher than I would expect based on my experience with variance in benchmarks.
I skimmed through lib.rs and didn't see "log" or "ln" so I think that may be the issue - linear regression assumes homoscedasticity and that assumption is violated by using a geometrically distributed number of iterations.
- https://en.wikipedia.org/wiki/Linear_regression#Assumptions
- https://en.wikipedia.org/wiki/Homoscedasticity
You should be able to fix it by using log-log regression - take the log of both iterations and elapsed time before computing the linear regression.