[BUG] - parametrizations tutorial no longer works in torch 2.0 (deprecated torch.solve API)

[murphyk]

Add Link

https://pytorch.org/tutorials/intermediate/parametrizations.html

Describe the bug

This fails

class CayleyMap(nn.Module):
    def __init__(self, n):
        super().__init__()
        self.register_buffer("Id", torch.eye(n))

    def forward(self, X):
        # (I + X)(I - X)^{-1}
        return torch.solve(self.Id + X, self.Id - X).solution #### Broken
        #return torch.linalg.solve(self.Id + X, self.Id - X)

layer = nn.Linear(3, 3)
parametrize.register_parametrization(layer, "weight", Skew())
parametrize.register_parametrization(layer, "weight", CayleyMap(3))
X = layer.weight
print(torch.dist(X.T @ X, torch.eye(3)))  # X is orthogonal

The offending line should be replaced by

return torch.linalg.solve(self.Id + X, self.Id - X)

Describe your environment

colab, CPU mode, torch 2.0.0+cu118

[murphyk]

Also need to replace print((torch.symeig(X).eigenvalues > 0.).all()) with print((torch.linalg.eigh(X).eigenvalues > 0.).all())

[murphyk]

And the "right inverse" section needs updating in the same way

class CayleyMap(nn.Module):
    def __init__(self, n):
        super().__init__()
        self.register_buffer("Id", torch.eye(n))

    def forward(self, X):
        # Assume X skew-symmetric
        # (I + X)(I - X)^{-1}
        #return torch.solve(self.Id + X, self.Id - X).solution
        return torch.linalg.solve(self.Id + X, self.Id - X)

    def right_inverse(self, A):
        # Assume A orthogonal
        # See https://en.wikipedia.org/wiki/Cayley_transform#Matrix_map
        # (X - I)(X + I)^{-1}
        #return torch.solve(X - self.Id, self.Id + X).solution
        return torch.linalg.solve(X - self.Id, self.Id + X)

[msaroufim]

Wanna make the PR? I can help merge