Error when quantizing Yolov5

[tucachmo2202]

Hi, I appreciate your contribute. I tried using quantization technique with your Aimet tool with yolov5 (version 5.0). When call function sim.compute_encodings() it throw the following error:

2022-05-11 21:36:46,119 - Quant - ERROR - Expecting quantize activation input of type torch.Tensor but got <class 'list'>
Traceback (most recent call last):
  File "yolov5_quantization.py", line 305, in <module>
    sim.compute_encodings(forward_pass_callback=pass_calibration_data, forward_pass_callback_args=500)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/quantsim.py", line 255, in compute_encodings
    _ = forward_pass_callback(self.model, forward_pass_callback_args)
  File "yolov5_quantization.py", line 265, in pass_calibration_data
    sim_model(inputs_batch)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/aimet/traffic_pytorch/models/yolo.py", line 124, in forward
    return self.forward_once(x, profile, visualize)  # single-scale inference, train
  File "/home/aimet/traffic_pytorch/models/yolo.py", line 156, in forward_once
    x = m(x)  # run
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/qc_quantize_op.py", line 347, in forward
    quantized_inputs = self._quantize_activation(self.input_quantizers, inputs)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/qc_quantize_op.py", line 447, in _quantize_activation
    raise AssertionError
AssertionError

. As Simon Cheng direction, I quantized yolov5 successfully. However, I found that many nodes skipped when quantize so the performance of int8 model is drop. Are there any guide to fix this error? Thank you very much!

[quic-ssiddego]

Hi @tucachmo2202 Thank you for reporting this. One way to debug this is to run the model validator to check if there are some updates needed to run quantization correctly : https://quic.github.io/aimet-pages/releases/1.21.0/api_docs/torch_model_validator.html#api-torch-model-validator. Prerequisites are captured here : https://quic.github.io/aimet-pages/releases/1.21.0/api_docs/torch_model_guidelines.html#api-torch-model-guidelines. After this, one could try model preparer to prepare the model for quantsim (example : perform auto replacement of functionals with nn modules etc) : https://quic.github.io/aimet-pages/releases/1.21.0/api_docs/torch_model_preparer.html#api-torch-model-preparer

[quic-akhobare]

Hi @tucachmo2202 - Latest version of AIMET 1.23.0 should handle Yolov5. Please try it out. I would follow the example here and use AIMET model preparer before creating the QuantSim model. https://github.com/quic/aimet/blob/develop/Examples/torch/quantization/qat.ipynb

Closing the ticket for now. If needed please re-open it.

[Dawsonqw]

Hi, I appreciate your contribute. I tried using quantization technique with your Aimet tool with yolov5 (version 5.0). When call function sim.compute_encodings() it throw the following error:

2022-05-11 21:36:46,119 - Quant - ERROR - Expecting quantize activation input of type torch.Tensor but got <class 'list'>
Traceback (most recent call last):
  File "yolov5_quantization.py", line 305, in <module>
    sim.compute_encodings(forward_pass_callback=pass_calibration_data, forward_pass_callback_args=500)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/quantsim.py", line 255, in compute_encodings
    _ = forward_pass_callback(self.model, forward_pass_callback_args)
  File "yolov5_quantization.py", line 265, in pass_calibration_data
    sim_model(inputs_batch)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/aimet/traffic_pytorch/models/yolo.py", line 124, in forward
    return self.forward_once(x, profile, visualize)  # single-scale inference, train
  File "/home/aimet/traffic_pytorch/models/yolo.py", line 156, in forward_once
    x = m(x)  # run
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/qc_quantize_op.py", line 347, in forward
    quantized_inputs = self._quantize_activation(self.input_quantizers, inputs)
  File "/usr/local/lib/python3.6/dist-packages/aimet_torch/qc_quantize_op.py", line 447, in _quantize_activation
    raise AssertionError
AssertionError

. As Simon Cheng direction, I quantized yolov5 successfully. However, I found that many nodes skipped when quantize so the performance of int8 model is drop. Are there any guide to fix this error? Thank you very much!

hi.Can you share a simple code that shows how to quantify yolov5 using aimet? When I call "model=prepare_model(model)" I get all kinds of errors. I don't know how to fix it.

[quic-hitameht]

Hello @haohao-qw AIMET model preparer uses torch FX symbolic trace to perform auto replacement of functional operations with nn type modules and inherits the limitations of torch FX symbolic trace. You might have to slightly modify the model definition in order to successfully prepare your model.

We have captured workarounds to resolve these limitations with code examples. Please let us know if this doesn't help.