James Chapman

@Michaelvll the change you have just made will break people's code I think! For the reasons I address above! Need to work round pytorch gradient sqrt(0) to fix the all...

I have an idea I'm just testing. I think given what I've just said we could do sum(sqrt(diag(T'T)))

Keeps the efficiency ie not using another eigendecomposition but hopefully won't break

Yep this looks like the one: torch.sum(torch.sqrt(torch.diag(torch.matmul(Tval.t(), Tval)))) Should do the job.

One thing to be aware of is that the gradients of this objective are pretty unstable as you reduce the batch size (basically because eigenvalue solvers are inexact so even...

Hi both, I've changed my mind on this. I now think that the use all singular values route is just wrong and would be better to push everything through topk...

Open to disagreement - I've got in touch with the first person to do it in this way 6 years ago which seems to be the foundation of (what I...

Yeah cool idea - would be delighted to accept a PR with essentially what you have described here. Otherwise will take a look in the future when I have some...

Have you got a citation for the calculation?

Yeah the examples are set up to work with pytorch lightning so should be as simple as passing GPUs=1 to trainer as in here: https://pytorch-lightning.readthedocs.io/en/stable/common/single_gpu.html