MaixPy-v1
MaixPy-v1 copied to clipboard
sipeed
[enhancement] Support for YOLOv3
This adds support for running multi-branch YOLO v3 object detection network on MaixPy. Number of branches can be set to 1 or 2. Support for larger number of branches is possible, but will likely bring sub-optimal performance (low FPS). The YOLO v3 model was trained using aXeleRate framework. It has number of advantages if compared to YOLO v2 models, that I was using before:
- multi-branch structure allows detecting small and large objects
- training is more stable, the model converges to lower loss
- YOLO v3 loss works better for non-square images
Below is the best results I was able to achieve with YOLO v2 224x224 MobileNet 0.75 backbone
And here is (probably) not the best YOLO v3, trained for only 50 epochs 320x240 MobileNet 0.5 backbone
The out comparison for YOLOv2 and YOLOv3
"""
a = kpu.init_yolo2(task, 0.2, 0.1, 5, anchor)
>>> {"x":20, "y":30, "w":10, "h":21, "value":0.204172, "classid":1, "index":0, "objnum":6}
{"x":195, "y":38, "w":25, "h":33, "value":0.388860, "classid":0, "index":1, "objnum":6}
{"x":96, "y":65, "w":24, "h":52, "value":0.280810, "classid":1, "index":2, "objnum":6}
{"x":60, "y":95, "w":25, "h":33, "value":0.388278, "classid":0, "index":3, "objnum":6}
{"x":150, "y":125, "w":25, "h":51, "value":0.287573, "classid":0, "index":4, "objnum":6}
{"x":17, "y":174, "w":25, "h":33, "value":0.388692, "classid":0, "index":5, "objnum":6}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
"""
"""
a = kpu.init_yolo3(task, 0.5, 0.3, 3, 2, anchor)
>>> kpu_net __del__
YOLO v3
{"x":205, "y":135, "w":47, "h":68, "value":0.935626, "classid":0, "index":0, "objnum":11}
{"x":101, "y":8, "w":15, "h":17, "value":0.732297, "classid":0, "index":1, "objnum":11}
{"x":160, "y":14, "w":16, "h":24, "value":0.980218, "classid":0, "index":2, "objnum":11}
{"x":32, "y":28, "w":19, "h":24, "value":0.969574, "classid":1, "index":3, "objnum":11}
{"x":110, "y":36, "w":20, "h":22, "value":0.635944, "classid":1, "index":4, "objnum":11}
{"x":202, "y":46, "w":22, "h":30, "value":0.978948, "classid":0, "index":5, "objnum":11}
{"x":261, "y":51, "w":28, "h":33, "value":0.983265, "classid":0, "index":6, "objnum":11}
{"x":103, "y":68, "w":21, "h":32, "value":0.620853, "classid":0, "index":7, "objnum":11}
{"x":131, "y":78, "w":38, "h":50, "value":0.831876, "classid":1, "index":8, "objnum":11}
{"x":83, "y":96, "w":30, "h":40, "value":0.648528, "classid":1, "index":9, "objnum":11}
{"x":21, "y":182, "w":30, "h":32, "value":0.926121, "classid":0, "index":10, "objnum":11}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
To maintain backwards compatibility, I have decided to preserve init_yolov2 and run_yolov2 functions and add init_yolov3 and run_yolov3 functions - this means code is somewhat clumsy, with nearly identical functions. Perhaps you have some suggestions or can make some edits yourself, @junhuanchen
I have compared the outputs for YOLOv2 inference on latest stable build of MaixPy and my fork, they are identical. Although more testing never hurts, if you'd like to do that.
"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)
kpu_net __del__
{"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-72-g22a8555b5 on 2021-11-01; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)
>>> {"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
That would take care of these threads: https://github.com/sipeed/MaixPy/issues/272 https://github.com/sipeed/MaixPy/issues/178 https://github.com/sipeed/MaixPy/issues/173
I am trying to upload a YOLOv3 script via MaixPy IDE to the M.A.R.K vehicle with k210 module but it seems it is still only compatible with YOLOv2. Will the master branch of the repo be merged upgraded to YOLOv3 anytime soon?
@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....
@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....
Dear, I’m very sorry that this project has stopped updating from the perspective of the enterprise and will be handed over to community users to take care of and maintain in the future. If you need it or are responsible for this submission, you can fork and test it to give reasons for the need to be merged.
The reason why I can‘t merge is not that it can’t work, but that I can‘t be responsible for this submission, because it also needs corresponding training scripts and document tutorials behind it to communicate to community users that this function is feasible, including API descriptions and sample code.
Forgive me for not being able to merge. In fact, I knew it was available for a long time, but I didn’t have the conditions to deal with what needs to be completed after merging it.
You know, we can‘t make money from this open source project. We can only earn income by selling hardware. We have gradually migrated to Linux AI chips in China. You can find this fact on our wiki.sipeed.com.
Thank you very much for your attention. If necessary, I can add you to the warehouse administrator to handle these PRs. If possible, please inform me of your reasons and expectations. I very much welcome our open source users to take the initiative to maintain and update the open source project.
Love you, junhuanchen.
Hi! Very thanks u pull requests it.I want to know how to use aXeleRate train my custom yolov3 model by this Repositories config list only have yolov2 configs.How can i modified the config just like detector.json