CenterNet MobileNetV2 - inference is too slow

[Paliking]

Hi,

I am able to run SSD MobileNetV2 and CenterNet MobileNetV2 (boxes prediction) on my android device. When I compare inference speed of the models on my android device I get below results:

inference of CenterNet MobileNetV2 512x512 is aprox. 5-6 times slower than SSD MobileNetV2 320x320 quantized model. inference of CenterNet MobileNetV2 512x512 is aprox. 3 times slower than SSD MobileNetV2 320x320 non-quantized model. inference of CenterNet MobileNetV2 512x512 has aprox. same speed as SSD MobileNetV2 640x640 quantized model.

I run the models on CPU only, using 4 threads based on the official android tflite tutorial (6 month old, hopefully it is not the reason). I have used CenterNet MobileNetV2 512x512 tflite model directly from here: http://download.tensorflow.org/models/object_detection/tf2/20210210/centernet_mobilenetv2fpn_512x512_coco17_kpts.tar.gz (which has btw input size 320x320 instead of 512x512) and I used also CenterNet MobileNetV2 512x512 tflite model created based on this tutorial: https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/centernet_on_device.ipynb both CenterNet MobileNetV2 512x512 tflite models have the same inference speed.

My understanding is the inference of CenterNet MobileNetV2 512x512 should be aprox. 3 times faster than SSD MobileNetV2 320x320 (based on official documentation/benchmark below)

Model name	Speed (ms)	COCO mAP	Outputs
CenterNet MobileNetV2 FPN 512x512	6	23.4	Boxes
SSD MobileNet v2 320x320	19	20.2	Boxes
SSD MobileNet V1 FPN 640x640	48	29.1	Boxes
SSD MobileNet V2 FPNLite 320x320	22	22.2	Boxes
SSD MobileNet V2 FPNLite 640x640	39	28.2	Boxes

What is your experience with inference speed of CenterNet MobileNetV2 512x512? Is it fast for you? Am I missing something? Why is CenterNet MobileNetV2 512x512 so slow for me?

thank you

[mui-bot]

	Messages
:book:	Netlify deploy preview: https://deploy-preview-8029--material-ui-x.netlify.app/

These are the results for the performance tests:

Test case	Unit	Min	Max	Median	Mean	σ
Filter 100k rows	ms	735.6	1,019.6	824.8	897.7	106.207
Sort 100k rows	ms	817	1,143.7	817	998.32	126.264
Select 100k rows	ms	207.4	317	299.2	283.54	40.187
Deselect 100k rows	ms	231.2	351.4	293.6	293.32	38.187

Generated by :no_entry_sign: dangerJS against e0b1dec571bee0ad1ed5219a82efcd19f2456b91