nltools icon indicating copy to clipboard operation
nltools copied to clipboard

Adjust sigma estimation for standard error computation in regress

Open ejolly opened this issue 6 years ago • 2 comments

In the specific situation where we estimate an intercept-free regression model, our current estimate of the variance of the residuals (np.std(res, axis=0, ddof=X.shape[1])) is biased and will yield slightly lower standard errors and slightly higher t-statistics (not true for models that contain an intercept). For this reason we should change our estimate of sigma to sigma = np.sqrt(res.T.dot(res) / (X.shape[0] - X.shape[1])), which doesn't have this issue.

ejolly avatar May 31 '19 06:05 ejolly

I think this is fine with me.

ljchang avatar Jun 20 '19 03:06 ljchang

Toyed with implementing this in https://github.com/cosanlab/nltools/pull/342 but ultimately didn't add it. Need to rethink the linear-algebra because my suggestion above assumed res is 1d, and if it's 2d, it blows up memory, likely due to the res.T.dot(res).

ejolly avatar Apr 15 '20 16:04 ejolly