Format error with ONNX quantized CNN

[mbelda]

Hello, I am trying to run a particular version of the MobilenetV2 CNN. I have quantized it and it can be seen in netron.app that it is correctly quantized using the QDQ format.

But when I try to execute the systolic_runner example minimaly modified (just the size of the labels array) on firesim I keep getting the following error:

terminate called after throwing an instance of 'Ort::Exception'
  what():  Unexpected input data type. Actual: (tensor(float)) , expected: (tensor(uint8))

I have tried to run the original Mobilenet_V2 quantized from the onnx repository and it works fine, but I have noticed that it is quantized with the operator oriented formal.

I attach my network. MobileNetV2_0p5_all_quant.zip

Do you support the QDQ format? Why is this happening?

Thank you in advance. Kind regards, MªJosé.

[hngenc]

Do you support the QDQ format?

By "the QDQ format", do you just mean placing QuantizeLinear and DequantizeLinear nodes in your model? If so, I believe we should support that.

Could you also share the image you used for the inference? I'm able to reproduce this error with your ONNX model, and the dog.jpg file in the ONNX-Runtime repo, but I'm not sure if you're using a different test image.

[mbelda]

It happens to me with the dog image too. You can use any image that fits on the expected measurements.

[hngenc]

Well, I think the dog image might just be failing because the first node in your ONNX graph expects int8 inputs, while the dog image is float. Perhaps you can insert another node which just casts the float image to int8, and then feed that into the rest of your ONNX model?