kotlindl
kotlindl copied to clipboard
Add NFResNet model to models package
The new family and approach of BatchNorm-free NN architectures look very perspective due to the lack of BatchNorm training support.
In the paper "High-Performance Large-Scale Image Recognition Without Normalization" the new approach is proposed.
We should apply this approach to the traditional ResNet architecture to prevent gradient exploding without BatchNorm layers.
The pre-trained weights will be a plus but are not nessesary.
How to obtain pre-trained weights https://reposhub.com/python/deep-learning/benjs-nfnets_pytorch.html
An example of pytorch implementation https://nfnets-pytorch.readthedocs.io/en/latest/