Network Pruning using Adaptive Exemplar Filters .

Tips

Any problem, please contact the first author (Email: [email protected]) or the third author (Email: [email protected]). Also, you can post issues with github, but sometimes we could not receive github emails thus may ignore the posted issues (sorry if it happens).

Pre-trained Models

We provide the pre-trained models used in our paper.

CIFAR-10

| VGG16 | ResNet56 | ResNet110 |GoogLeNet |

ImageNet

| ResNet18 | ResNet34 | ResNet50 | ResNet101 | ResNet152 |

Result Models

We provide our pruned models in the experiments, along with their training loggers and configurations.

CIFAR-10

ImageNet

Running Code

The code has been tested using Pytorch1.3 and CUDA10.0 on Ubuntu16.04.

Requirements: Sklearn 0.20.1

EPruner

You can run the following code to search model on CIFAR-10:

python epruner_cifar.py 
--dataset cifar10 
--data_path /data/CIFAR10/ 
--pretrain_model /data/model/resnet56.pt 
--job_dir /data/experiment/resnet56 
--arch resnet 
--cfg resnet56 
--init_method centroids 
--preference_beta 0.76 
--lr 0.01 
--lr_decay_step 50 100 
--num_epochs 150 
--train_batch_size 256 
--weight_decay 5e-3 
--gpus 0

You can run the following code to search model on ImageNet:

python epruner_imagenet.py 
--dataset imagenet 
--data_path /data/ImageNet/ 
--pretrain_model /data/model/resnet50.pth 
--job_dir /data/experiment/resnet50 
--arch resnet 
--cfg resnet50 
--init_method centroids 
--preference_beta 0.75 
--lr 0.1 
--lr_decay_step 30 60 
--num_epochs 90 
--train_batch_size 256 
--weight_decay 1e-4 
--gpus 0 1 2 

Test Our Performance

Before you testing our model, please use the following command to install the thop python package which can calculate the flops and params of model:

pip install thop

Follow the command below to verify our pruned models:

python test_flops_params.py
--dataset cifar10 
--data_path /data/CIFAR10 
--arch resnet 
--cfg resnet56
--pruned_model /data/model/pruned_model/resnet56/model_best.pt 
--eval_batch_size 100
--gpus 0

Other Arguments

optional arguments:
  -h, --help            show this help message and exit
  --gpus GPUS [GPUS ...]
                        Select gpu_id to use. default:[0]
  --dataset DATASET     Select dataset to train. default:cifar10
  --data_path DATA_PATH
                        The dictionary where the input is stored.
                        default:/home/lishaojie/data/cifar10/
  --job_dir JOB_DIR     The directory where the summaries will be stored.
                        default:./experiments
  --arch ARCH           Architecture of model. default:resnet
  --cfg CFG             Detail architecuture of model. default:resnet56
  --num_epochs NUM_EPOCHS
                        The num of epochs to train. default:150
  --train_batch_size TRAIN_BATCH_SIZE
                        Batch size for training. default:256
  --eval_batch_size EVAL_BATCH_SIZE
                        Batch size for validation. default:100
  --momentum MOMENTUM   Momentum for MomentumOptimizer. default:0.9
  --lr LR               Learning rate for train. default:1e-2
  --lr_decay_step LR_DECAY_STEP [LR_DECAY_STEP ...]
                        the iterval of learn rate. default:50, 100
  --weight_decay WEIGHT_DECAY
                        The weight decay of loss. default:5e-4
  --pretrain_model PRETRAIN_MODEL
                        Path to the pretrain model . default:None
  --init_method INIT_METHOD
                        Initital method of pruned model. default:centroids.
                        optimal:random,centroids,random_project
  --preference_beta PREFERENCE_BETA
                        The coefficient of preference used in
                        AffinityPropagation cluster. default:0.75