PyTorch-FilterResponseNormalizationLayer
PyTorch-FilterResponseNormalizationLayer copied to clipboard
yukkyo
PyTorch implementation of "Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks"
PyTorch-Filter Response Normalization Layer(FRN)
PyTorch implementation of Filter Response Normalization Layer(FRN)
0. How to apply FRN to your model
Replace BatchNorm2d + ReLU in the model with FRN + TLU yourself.
Currently, it is difficult to easily replace them with functions.
Because many models use the same ReLU in various places.
1. Experiment(Classification)
We use Best Artworks of All Time | Kaggle dataset.
This dataset contains 49 artists and their pictures.
In this experiment, we classify artist by picture.
1.0 Assumed libraries
- torch==1.3.1
- catalyst==19.11.6
- albumentations==0.4.3
- NVIDIA/apex
- If you use
--fp16option
- If you use
1.1 Get dataset
If you can use kaggle API command, you can download easily
$ cd input
$ kaggle datasets download -d ikarus777/best-artworks-of-all-time
$ unzip best-artworks-of-all-time.zip -d artworks
Or download directly from Best Artworks of All Time | Kaggle
I assume the following directory structure.
input
├── artworks
│ ├── artists.csv
│ ├── images
│ │ └── images
│ │ ├── Alfred_Sisley
│ │ │ ├── Alfred_Sisley_1.jpg
│ │ │ ├── Alfred_Sisley_10.jpg
│ │ │ ├── ...
1.2 Train(and Valid)
You can use --fp16 if you installed nvidia/apex.
But FRN is not tuned for FP16, you should turn off --fp16 when use --frn.
$ python train_cls.py --fp16
$ python train_cls.py --frn
yukkyo