1x1 group convs in LiteMLA

[benjamindorra]

Hello, thank you for the great work and code !

I have been looking at the LiteMLA block code and I'm not sure what the 1x1 group convolutions are for.

The paper does not seem to explain this much either.

With 1 parameter for every channel this cannot be too powerful, my only hypothesis is that this helps with channel scaling.

Do you have details on this operation ?