Model unstable when using transformation

[hilmar05]

Hello @Michaelvll, @arminarj

I am using the DeepCCA objective in a variational autoencoder setting. I use U, D, Vh = torch.linalg.svd(Tval) so i can getthe U and Vh together with SigmaHat11RootInv and SigmaHat22RootInv to get my transformation matrices w1 and w2, similar as in the file linear_cca.py but i used pytorch for the implementation. When i apply the transformation to my original space, the model has trouble learning. Without the transformation, there is no problem. I suspect unstable gradient as stated here under the warning tab because of the singular values. https://pytorch.org/docs/stable/generated/torch.linalg.svd.html#torch.linalg.svd.

Does anyone have an idea how to solve this problem?

Thanks you.

[arminarj]

Hi @hilmar05,

Thanks for reaching out. Have you tried adding some small values to Tval Matrix prior to using the SVD operation, just as it is implemented in this line of the objective file?

[hilmar05]

Hello @arminarj

Yes. I do the same in my code. Adding it improves the model definitely. But it is still not as good as without transformation, which i think should be possible because the transformation does not change the structure of the data. I use torch.eye right now. Maybe i need to optimize the parameter for regularisation or try some other methods. Do you have any ideas?