Moore-Penrose pseudo-inverse in linear regression

[liadbiz]

Hi, I am reimplementing ml algorithms based on yours

But I am a little confused about the part of the calculation of Moore-Penrose pseudoinverse in linear regression.

https://github.com/eriklindernoren/ML-From-Scratch/blob/40b52e4edf9485c4e479568f5a41501914fdc55c/mlfromscratch/supervised_learning/regression.py#L111-L114

Why do not usenp.linalg.pinv(X), according to the docstring:

It can compute the Moore-Penrose pseudo-inverse directly.

Thanks!

[Tangjacson]

i thought both were right, and pinv is more efficient. and i thought the third row(113) should be V.T.dot(...) maybe a typo?

[JonathanBechtel]

i thought both were right, and pinv is more efficient. and i thought the third row(113) should be V.T.dot(...) maybe a typo?

I looked into it and you are correct. Numpy doesn't return the regular SVD values. It returns values such that A = U @ S @ V, but normal SVD is A = U @ S @ V.T, which means if you're going to use V in later calculations you need to use its transpose.

I double checked the results and they're incorrect in their current form. I submitted a pull request to fix the issue.

[JonathanBechtel]

Also, for what it's worth, I think the current solution in place is a bit more 'expressive' than just using np.linalg.pinv(X). They both ultimately do the same thing, but assuming the point of this repo is to communicate how ML techniques work, then the current format seems better IMO.