Exposed group norm layer

[lostmsu]

[lostmsu]

PS. I simply took batch_norm.rs and rewrote for group norm. Without exact understanding what group norm is supposed to do.

[LaurentMazare]

What is the typical use case for this? It would be good to check that it's commonly used with this kind of api, ideally if you can add some example model that relies on this and where it improve performance that would be great.

[lostmsu]

I faced it when developing a port of Wave U-Net. Do you want the whole thing in tch-rs? I was thinking to make a separate crate.

[LaurentMazare]

Indeed this seems like a large model so a separate crate sounds good. I would suggest first experimenting with the group norm layer in this crate to see if the api fits properly the use case and only later move it to the tch crate if it's general enough (the api you suggested definitely seems to be general enough but I'm unsure whether it would work well for models using group norm and breaking the api later is a bit of a pain, so using it in some actual model is probably the best way to go).

[LaurentMazare]

Cleaning up old entries, some group-norm layer was added recently, hopefully it will cover your use cases. Feel free to open an issue if not the case.