Initial support for YOLOv3-tiny, 3x performance on edge TPU

[no2chem]

This PR adds support for YOLOv3-tiny, which increases the detection fps vs mobiledet from about 9fps to 27 fps on my Edge TPU. At least in my tests, accuracy seems to be pretty similar, if not slightly better.

The model used is located here.

There are some differences in the tensor output of the YOLO models, so I've refactored things out to a separate function. Hopefully that makes adding different models easier in the future. I've taken a large chunk of code from @guichristmann's repository and adapted it. I'm not exactly a numpy wizard, so there's some stuff left over relating to image scaling that can probably be removed.

I've added some documentation on how to use YOLOv3-tiny as well.

You'll need to map the model in via a volume mount, and use the following model configuration:

model:
  # Required: height of the trained model
  height: 416
  # Required: width of the trained model
  width: 416
  # Required: type of model (ssd or yolo)
  model_type: 'yolo'
  # Required: path of label map
  label_path: '/labelmap.txt'
  # Optional: (but required for yolo) - anchors, comma separated
  anchors:  '10,14,  23,27,  37,58,  81,82,  135,169,  344,319'

I'm working on trying to quantize YOLOv4-tiny to get that working on the Edge TPU next.

[digiblur]

Nice! I'll have to check this one out!

[rogerquake]

Just tried it out, and didn't detect me right away even when I was pretty close in the daylight. Eventually it got me, but it took way longer than the built-in models. Any ideas for other tweaks that are required to make this function properly?

[no2chem]

Interesting, @rogerquake

It probably is functioning properly, I assume you were running on the Edge TPU? What kind of detection rates were you getting?

I'm checking out the mAP in an attempt to quantify the difference objectively.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[extremeshok]

Any progress with this PR ? What needs to be done to have it as an option ? ie user can select yolo or mobiledect

[jonwilliams84]

I like the idea of this...would also be useful to enable it to automatically switch to a Night Time / IR optimised model when a camera switches to b/w to get better accuracy at night?

[ozett]

would be great to see detection-results side-by-side from yolo3 vs tensoflow-lite . on my experiments yolo did also slightly better on static images. would be interesting to see the fps on video-streams.. 👍 (to not forget to integrate this)

[andrempo]

Do you think this would be better for CPU too?

[ozett]

Do you think this would be better for CPU too?

i did not measure my yolo code on CPU, but ist running over a year and did not notice any burdon on the CPU, albeit its only doing inference on motions sequences, not constantly.

but if i have wild guess, i would think its resource-friendly...

[gururise]

Very excited about YoloV4-tiny, but really curious to try Yolo V3

[osos]

yolov5 ? https://blog.roboflow.com/yolov5-v6-0-is-here/ https://github.com/ultralytics/yolov5/

[ozett]

Very excited about YoloV4-tiny, but really curious to try Yolo V3

do you have yolov4-tiny running with frigate? some instructions to get started? greatly interested to to try this in comparison...

[ozett]

YOLOv5 could somehow run as an edgeTPU model now... anybody got into this?

https://github.com/ultralytics/yolov5/pull/3630#issuecomment-976300814

edit: seems that there are precomiled model available:

https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5l.mlmodel

[triptec]

@ozett there's also a export.py in the repo. I'm currently moving from pytorch+yolov5s to frigate. I'm only using cpu for detection and would really like to yolo instead because the difference in detection between ssd and yolov5 for my use case(detect cats/dogs) feels quite big (I've only done detection on the whole frame with yolo so I'm interested to see what happens if it only got the area of interest.

@blakeblackshear any thoughts on this PR or on how to be able to run other models like yolo for example?

[ozett]

yolo v5 with pytorch is a heavy-weight champion. maybe not so easy to integrate like yolo v3/v4

i had some times with it for using birds-detection in doods2 https://github.com/snowzach/doods2/issues/38

[triptec]

@ozett actually I've managed to integrate yolov5s with smaller images. I don't have a tpu so don't know if that works but I'm getting detection fps around 5-9. And currently it seems one can't event get a hold of a usb tpu.

[ozett]

@ozett actually I've managed to integrate yolov5s

thats great news. did you documented it anywhere? github? do you still have the functionality of frigate post-detections? i thought that responseformats or detections-messages by another modell type may break frigate. but if that works, i may tray to use that too? I have another spare maschine with a TPU for that available..

sounds great to try this, and to try to bring my custom.trained yolov5 model for our cats, which is running fine stand-alone, than into frigate...

[triptec]

@ozett actually I've managed to integrate yolov5s

thats great news. did you documented it anywhere? github? do you still have the functionality of frigate post-detections? i thought that responseformats or detections-messages by another modell type may break frigate. but if that works, i may tray to use that too? I have another spare maschine with a TPU for that available..

sounds great to try this, and to try to bring my custom.trained yolov5 model for our cats, which is running fine stand-alone, than into frigate...

I did it last night and the code is in a deplorable state but it works on the cpu at least but should work on a tpu as well. Post-detection works but you are correct that both input and output to/from the model needs to be changed, much like in this branch which is what I used as a starting point.

I might have some time tonight to at least remove pytorch and stuff that I used in the beginning. I've tried many ways, some of them not very clever ;)

If I get a branch working when will you have time to try it?

Also my work builds on this branch and it hasn't been merged for some reason and that makes me hesitant to spend much time on making my work presentable. Could @blakeblackshear chime in with feedback on the overall structure of adding type and other things to model config?

[triptec]

@ozett here you go =)

[blakeblackshear]

This branch is so old that these changes would need to be rewritten on current code.

[ozett]

@triptec here you use yolov5 as a converted .tflite model? not in its full form with the whole pytorch stuff?

(yolov3 was simple small, but yolov5 seems different, and yolov5.tflite is an "converted" export. just for clarification of the differences of the final model files)

[triptec]

@ozett correct, the models linked are exported versions of yolov5 that runs on tensorflow-lite without any pytorch stuff.

(yolov3 was simple small, but yolov5 seems different, and yolov5.tflite is an "converted" export. just for clarification of the differences of the final model files)

I'm not sure what you mean. But I think yolov3 was built with tensorflow while yolov5 was built with pytorch. I think for any of them to work on tensorflow-lite/tpu they need to be converted. But I am no expert. I'm also not sure by what measure yolov3 was small and simple where yolov5 is not. I think it depends on what version of the models you select.

[svenstaro]

@blakeblackshear same as #4532 this is way out of date and should be closed.