kotlindl icon indicating copy to clipboard operation
kotlindl copied to clipboard

Add NFResNet model to models package

Open zaleslaw opened this issue 3 years ago • 2 comments

The new family and approach of BatchNorm-free NN architectures look very perspective due to the lack of BatchNorm training support.

In the paper "High-Performance Large-Scale Image Recognition Without Normalization" the new approach is proposed.

We should apply this approach to the traditional ResNet architecture to prevent gradient exploding without BatchNorm layers.

The pre-trained weights will be a plus but are not nessesary.

zaleslaw avatar May 18 '21 09:05 zaleslaw

How to obtain pre-trained weights https://reposhub.com/python/deep-learning/benjs-nfnets_pytorch.html

zaleslaw avatar Dec 15 '21 11:12 zaleslaw

An example of pytorch implementation https://nfnets-pytorch.readthedocs.io/en/latest/

zaleslaw avatar Dec 15 '21 11:12 zaleslaw