Is there way to apply this to regular Linear layers

[JanoschMenke]

[prlz77]

I have not tried but you could just change this: https://github.com/prlz77/orthoreg/blob/master/orthoreg_pytorch.py#L18, and reshape the linear layer to have the same shape as a 1x1 convolution (for compatibility with the current code). That would be size out x in x 1 x 1. If you try and it works, please let me know :)

[JanoschMenke]

Actually the only adaption you need it if if type(m) == torch.nn.Linear: A out x in x 1 x 1 will result in a out x in matrix when applying m.weight.data.clone().view(m.weight.size(0), -1)

[prlz77]

Right! 👍