about quantized_model.onnx to quantized.dlc

[huxian0402]

Hi,

I use aimet_pytorch, and the generated files are as follows: quantized_model.pth, quantized_model.encodings.yaml, quantized_model.onnx, quantized_model.encodings, temp_onnx_model_with_markers.onnx．

I am having problems deploying the quantized_model.onnx and quantized_model.encodings generated by aimet to snpe. The commands used are as follows: snpe-onnx-to-dlc -i aimet_quantized_model.onnx --quantization_overrides quantized_model.encodings -o float_model_with_quant_params.dlc

snpe-dlc-quantize --input_dlc float_model_with_quant_params.dlc --input_list ../ref-imgs-new/raw_lmk_list_new.txt --output_dlc aimet_quantized_model_quantized.dlc --enable_htp --override_params

The quantization parameters of aimet_quantized_model_quantized.dlc seen with snpe-dlc_info are abnormal, The activation layer is normal 8-bit quantization, and the quantization parameters are the same as quantized_model.encodings; but the weight display of the convolutional layer is 32-bit, weight encoding: min -0.029826, max 0.029826, delta 0.000000, offset -2147483648.000000 bitwidth 32

Is this a bug?

@mohanksriram @Rohan-Chaudhury @aimetci @quic-bharathr @quic-mangal

[quic-ssiddego]

Hi @huxian0402 Thank you for the query. Could you please share all the output files along with encodings json file that was generated in this case? One thing is to check is if the quantization was skipped for this particular layer in your model. You could quickly check that using some sample code as shown in this test: https://github.com/quic/aimet/blob/develop/TrainingExtensions/torch/test/python/test_quantsim_config.py#L93.

[LLNLanLeN]

@huxian0402 May I ask which version of AIMET you're using. I believe the INT32 issue might be fix with latest AIMET (i'm not sure, I'm still using 1.14.0, but I've seen some MRs to address this issue).

If you're using 1.14.0 or older, the value is actually in bidwidth 32 (same ask mine) when you enable override_params. What I did was disable this option, instead pass in the text files that contains data for SNPE quantization process. In my personal experience, you'll get similar results if you do SNPE quantize compare to override_params.

[alexuechao]

@huxian0402 Hi，I meet the same issue recently, Did you solve this bug?

[quic-mangal]

Closing this issue due to inactivity. Please re-open it/ create a new issue if you need further help.