Quantization.MXNet
Quantization.MXNet copied to clipboard
hey-yahei
一些问题...
您好，我正在阅读并测试您的代码，我有一些疑惑和问题想要请教您。 我想复现white paper里mobileNetv2的per-channel量化，结果在imageNet val上面测试结果只有0.09%，说明per-channel的方案完全没有效果，请问您有什么头绪吗？
************************* Settings ************************* eval_per_calib : False use_gpu : 2 batch_norm : False use_gn : False input_bits_width : 8 exclude_first_conv : true dataset : imagenet batch_size : 64 use_se : False print_model : False input_signed : True list_models : False disable_cudnn_autotune : False last_gamma : False num_workers : 6 num_sample : 5 calib_mode : kl quantize_input_offline : True fixed_random_seed : 7 merge_bn : True calib_epoch : 3 model : mobilenetv2_1.0 quant_type : channel weight_bits_width : 8
会不会是因为weight量化方案采用max对称量化的关系？我看depth wise convulation上面权重的异常值比较多，导致max对称量化后信息损失太大？
不好意思，时隔一个多月今天才看到这个issue。经过排查发现是input的量化有问题，先前都只考虑了relu+conv的情况，没有考虑负值inputs，现在已经更新了代码。但是，
- 目前还无法为mbv2使用KL校准，主要原因是np.bincount不支持负值，目前还没来得及解决这个问题，后续会处理一下；
- mbv2在在线量化的表现还行，但naive校准的离线量化效果不佳，可能是input的数值分布差异比较大导致的。
$ python simulate_quantization.py --model=mobilenetv2_1.0 --use-gpu=1 --merge-bn --quant-type=channel --input-signed=true
************************* Settings ************************* model : mobilenetv2_1.0 print_model : False list_models : False use_gpu : 1 dataset : imagenet use_gn : False batch_norm : False use_se : False last_gamma : False merge_bn : True weight_bits_width : 8 input_signed : true input_bits_width : 8 quant_type : channel num_workers : 4 batch_size : 128 num_sample : 5 quantize_input_offline : False calib_mode : naive calib_epoch : 3 disable_cudnn_autotune : False eval_per_calib : False exclude_first_conv : true fixed_random_seed : 7
************************* Exclude blocks ************************* mobilenetv20_features_conv0 mobilenetv20_features_batchnorm0 mobilenetv20_output_pred
Eval: 0%| | 0/391 [00:00<?, ?it/s][15:30:48] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) Eval: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌| 390/391 [03:02<00:00, 2.60it/s][15:33:52] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) Eval: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 391/391 [03:05<00:00, 1.21s/it] ************************* Result ************************* acc : 71.99% avg_acc : 71.99%
$ python simulate_quantization.py --model=mobilenetv2_1.0 --use-gpu=1 --merge-bn --quant-type=channel --input-signed=true --quantize-input-offline
************************* Settings ************************* model : mobilenetv2_1.0 print_model : False list_models : False use_gpu : 1 dataset : imagenet use_gn : False batch_norm : False use_se : False last_gamma : False merge_bn : True weight_bits_width : 8 input_signed : true input_bits_width : 8 quant_type : channel num_workers : 4 batch_size : 128 num_sample : 5 quantize_input_offline : True calib_mode : naive calib_epoch : 3 disable_cudnn_autotune : False eval_per_calib : False exclude_first_conv : true fixed_random_seed : 7
************************* Exclude blocks ************************* mobilenetv20_features_conv0 mobilenetv20_features_batchnorm0 mobilenetv20_output_pred
************************* Naive Calibration ************************* Calib[1/3]: 0%| | 0/40 [00:00<?, ?it/s][15:45:32] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) Calib[1/3]: 98%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 39/40 [00:21<00:00, 2.46it/s][15:45:53] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) Calib[1/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:23<00:00, 1.28it/s] Calib[2/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:18<00:00, 3.05it/s] Calib[3/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:18<00:00, 3.12it/s] Best threshold for mobilenetv20_features_linearbottleneck0_conv0: 5.988860607147217 Best threshold for mobilenetv20_features_linearbottleneck0_conv1: 5.999978542327881 Best threshold for mobilenetv20_features_linearbottleneck1_conv0: 18.790611267089844 Best threshold for mobilenetv20_features_linearbottleneck1_conv1: 5.999978542327881 Best threshold for mobilenetv20_features_linearbottleneck1_conv2: 5.984673023223877 Best threshold for mobilenetv20_features_linearbottleneck2_conv0: 13.01661205291748 Best threshold for mobilenetv20_features_linearbottleneck2_conv1: 5.990994453430176 Best threshold for mobilenetv20_features_linearbottleneck2_conv2: 5.999978542327881 Best threshold for mobilenetv20_features_linearbottleneck3_conv0: 18.15996742248535 Best threshold for mobilenetv20_features_linearbottleneck3_conv1: 5.973598957061768 Best threshold for mobilenetv20_features_linearbottleneck3_conv2: 5.948035717010498 Best threshold for mobilenetv20_features_linearbottleneck4_conv0: 12.318225860595703 Best threshold for mobilenetv20_features_linearbottleneck4_conv1: 5.672513961791992 Best threshold for mobilenetv20_features_linearbottleneck4_conv2: 5.92328405380249 Best threshold for mobilenetv20_features_linearbottleneck5_conv0: 14.465754508972168 Best threshold for mobilenetv20_features_linearbottleneck5_conv1: 5.8163628578186035 Best threshold for mobilenetv20_features_linearbottleneck5_conv2: 5.846293926239014 Best threshold for mobilenetv20_features_linearbottleneck6_conv0: 17.13546371459961 Best threshold for mobilenetv20_features_linearbottleneck6_conv1: 5.877142906188965 Best threshold for mobilenetv20_features_linearbottleneck6_conv2: 5.84243106842041 Best threshold for mobilenetv20_features_linearbottleneck7_conv0: 9.198147773742676 Best threshold for mobilenetv20_features_linearbottleneck7_conv1: 4.9297709465026855 Best threshold for mobilenetv20_features_linearbottleneck7_conv2: 5.560483932495117 Best threshold for mobilenetv20_features_linearbottleneck8_conv0: 10.592511177062988 Best threshold for mobilenetv20_features_linearbottleneck8_conv1: 5.073105335235596 Best threshold for mobilenetv20_features_linearbottleneck8_conv2: 5.82119083404541 Best threshold for mobilenetv20_features_linearbottleneck9_conv0: 11.359800338745117 Best threshold for mobilenetv20_features_linearbottleneck9_conv1: 5.044006824493408 Best threshold for mobilenetv20_features_linearbottleneck9_conv2: 5.929920196533203 Best threshold for mobilenetv20_features_linearbottleneck10_conv0: 11.696471214294434 Best threshold for mobilenetv20_features_linearbottleneck10_conv1: 5.494167804718018 Best threshold for mobilenetv20_features_linearbottleneck10_conv2: 5.944950103759766 Best threshold for mobilenetv20_features_linearbottleneck11_conv0: 9.514467239379883 Best threshold for mobilenetv20_features_linearbottleneck11_conv1: 5.645473957061768 Best threshold for mobilenetv20_features_linearbottleneck11_conv2: 5.95599889755249 Best threshold for mobilenetv20_features_linearbottleneck12_conv0: 10.617852210998535 Best threshold for mobilenetv20_features_linearbottleneck12_conv1: 5.6360182762146 Best threshold for mobilenetv20_features_linearbottleneck12_conv2: 5.9353556632995605 Best threshold for mobilenetv20_features_linearbottleneck13_conv0: 12.687278747558594 Best threshold for mobilenetv20_features_linearbottleneck13_conv1: 5.4570746421813965 Best threshold for mobilenetv20_features_linearbottleneck13_conv2: 5.798008441925049 Best threshold for mobilenetv20_features_linearbottleneck14_conv0: 7.8866963386535645 Best threshold for mobilenetv20_features_linearbottleneck14_conv1: 5.743549823760986 Best threshold for mobilenetv20_features_linearbottleneck14_conv2: 5.802321434020996 Best threshold for mobilenetv20_features_linearbottleneck15_conv0: 9.552735328674316 Best threshold for mobilenetv20_features_linearbottleneck15_conv1: 5.421483516693115 Best threshold for mobilenetv20_features_linearbottleneck15_conv2: 5.760587215423584 Best threshold for mobilenetv20_features_linearbottleneck16_conv0: 11.939920425415039 Best threshold for mobilenetv20_features_linearbottleneck16_conv1: 5.075140953063965 Best threshold for mobilenetv20_features_linearbottleneck16_conv2: 5.953917503356934 Best threshold for mobilenetv20_features_conv1: 5.507375240325928
Eval: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 391/391 [03:05<00:00, 1.21s/it] ************************* Result ************************* acc : 65.17% avg_acc : 65.17%
