Why does quantified model run slower?

[oroelipas]

I know these models are designed to run in Coral USB but should't also it run faster in a PC?

It takes about 1.15 seconds to run a tiny-yolov3.tflite in my coumputer but arround 15 seconds to run the quant_coco-tiny-v3-relu.tflite

Is this normal behaviour? will it run faster than 1.15s in Coral USB TPU when I buy it?

Thanks for answering!

[guichristmann]

I don't know the reason, but I also got a bad performance when running the quantized model on the desktop CPU. However, it runs fine/as expected with the Edge TPU.

[parthjdoshi]

@guichristmann What kind of inference speeds did you get with YOLOv3-tiny? In terms of FPS?

[AkkiSony]

@oroelipas @parthjdoshi @guichristmann Hi. I could convert my yolov3 model to quantized tflite model. But when I try to run the inference.py script. I do not see any detection happening on the input image. Were you able to get it doneusing this platform? Did you make any changes in the script? Please do help me get this run. Thank you in advance guys.

[lukqw]

@parthjdoshi I get around 16-17 fps with a yolov3-tiny model with relu as the activation function on device with edge tpu