Has anyone successfully gain accuracy after quantification on vgg16?

[blueardour]

I tried 50% log quantization on the pre-trained vgg16, however failed to re-gain the original accuracy. Have anyone successful with the experiments?
Any suggestions on how to re-gain the accuracy are welcome.

[AojunZhou]

how about pre-trained model performance and the 50% quantization performance?

[blueardour]

In my experiment, origin accuracy of VGG16 on imagenet evaluation dataset is (top1/top5: 0.6835/0.88442).

After 50% log quantization, the top1/top5 accuracy became to (0.63682/0.85252). The accuracy kept the same even after 10 epochs of iterations. My solver.prototxt looked like the following:

net: "vgg16_train.prototxt" test_iter: 6250 test_interval: 5000 test_initialization: true

base_lr: 0.00005 lr_policy: "step" stepsize: 10000 gamma: 0.5 momentum: 0.9 weight_decay: 0.0005

iter_size: 1 max_iter: 37000000

display: 50 snapshot: 5000 snapshot_prefix: "./models/quant/vgg16" solver_mode: GPU

Besides, I noticed a tip of shuffle data at runtime was suggested, thus I employed ImageData layer with shuffle option to be true in the vgg_train.prototxt.

transform_param { crop_size: 224 mean_value: 104 mean_value: 117 mean_value: 123 mirror: true } image_data_param { source: "/workspace/data/imagenet/caffe-ilsvrc/train.txt" root_folder: "/workspace/data/imagenet/ILSVRC2012_img_train/" batch_size: 64 shuffle: true new_height: 256 new_width: 256 }

Any suggestions?

[blueardour]

I am trying to only quantize the last three fc layers and keep conv layers by set layer level lr to be 0. Fine-tuning only happens on FC layers. Wish luck with me

[blueardour]

Hi, @Zhouaojun I met some problem when re-implement your experiments as described above. The accuracy seemed not re-gain after quantization. I wonder if it is convenient for you to share the fine-tuning log file (any example is welcome) so that I could has a sense of successful quantization procedure and analyse why I'm going wrong?