Switchable-Whitening

Switchable-Whitening copied to clipboard

Switchable Whitening (SW)

Paper

Xingang Pan, Xiaohang Zhan, Jianping Shi, Xiaoou Tang, Ping Luo. "Switchable Whitening for Deep Representation Learning", ICCV2019.

Introduction

• Switchable Whitening unifies various whitening and standardization techniques in a general form, and adaptively learns their importance ratios for different tasks.
• This repo is for ImageNet classification. We also provide the code for Syncronized SW at models/ops/sync_switchwhiten.py, which could be used for detection and segmentation.

Requirements

• python>=3.6

• pytorch>=1.0.1

• others

  pip install -r requirements.txt
  

Results

Top1/Top5 error on the ImageNet validation set are reported. The pretrained models with SW are available at google drive or BaiduYun(password: xkdi).

Before Start

1. Clone the repository

   git clone https://github.com/XingangPan/Switchable-Whitening.git
   

2. Download ImageNet dataset. You may follow the instruction at fb.resnet.torch to process the validation set.

Training

1. Train with nn.DataParallel

   sh experiments/resnet50_sw/run.sh  # remember to modify --data to your ImageNet path
   

2. Distributed training based on slurm

   sh experiments/resnet50_sw/run_slurm.sh ${PARTITION}
   

Practical concerns

• Inspired by IterativeNorm, SW is accelarated via Newton's iteration.
• For SW, 4x64 (GPU number x batchsize) performs slightly better than 8x32.

Citing SW

@inproceedings{pan2018switchable,
  author = {Pan, Xingang and Zhan, Xiaohang and Shi, Jianping and Tang, Xiaoou and Luo, Ping},
  title = {Switchable Whitening for Deep Representation Learning},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
  month = {October},
  year = {2019}
}