Closed Form Linear Regression

[Heyjuke58]

Hello!

I stumbled upon what i think might be a typo in the paper:

In the lower part (the formula for the optimal P) i believe the second y in the numerator should also be transposed.

Lets say dimensions of X are ( i, j ) then the dimensions of y should be ( i, 1 ). X^t * y should work fine giving z having dimensions ( j, 1 ). Then z cannot be multiplied with y (( j, 1) * ( i, 1 )), but with y^t (( j, 1 ) * ( 1, i ) = ( j, i )).

I believe you have not used this formula in your code or experiments. For someone who might want to do that here is what i wrote:

@staticmethod
def rlace_linear_regression_closed_form(X: torch.Tensor, y: torch.Tensor):
    return torch.eye(X.shape[1], X.shape[1]) - (
            (X.t().mm(y)).mm(y.t().mm(X)) / ((y.t().mm(X)).mm(X.t().mm(y)))
    )