boxmot
boxmot copied to clipboard
mikel-brostrom
Adding Bot sort Tracker
As discussed in the previous PR, BoT-SORT tracker is implemented using the ReID architectures used in the repo.
Also have made some changes in the README.md file for the installation of cython_bbox.
Demo video:
https://user-images.githubusercontent.com/82194525/209459332-e1c74fca-25b6-4b1f-8cdb-be8338b40777.mp4
Nice :rocket:! I am evaluating botsort now on MOT17. Dropping the results in a few minutes.
Btw, I got rid of pximport so that no special installation is needed for botsort.
Nice 🚀! I am evaluating botsort now on MOT17. Dropping the results in a few minutes. Btw, I got rid of
pximportso that no special installation is needed for botsort.
That's great, please keep me updated.
But the processing is quite slow, I have checked out with original BoT-SORT repo as well.
HOTA: exp349-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 59.966 59.194 61.305 63.979 76.757 65.175 81.042 81.192 62.57 79.29 75.634 59.971
MOT17-05-FRCNN 41.459 40.198 42.893 42.643 76.733 49.454 70.329 82.231 42.755 52.564 77.796 40.892
MOT17-09-FRCNN 56.251 62.262 50.854 67.254 81.578 60.388 73.005 85.758 58.47 69.636 82.087 57.162
MOT17-10-FRCNN 50.496 50.887 50.386 54.443 75.73 55.301 74.632 80.615 52.355 66.507 75.906 50.483
MOT17-11-FRCNN 62.056 60.486 63.862 71.29 74.669 71.946 80.615 86.878 67.463 74.138 83.724 62.071
MOT17-13-FRCNN 48.22 43.401 53.945 46.486 74.637 60.091 74.942 80.696 50.031 62.769 75.813 47.587
COMBINED 56.389 54.99 58.369 59.847 76.449 63.768 79.252 82.086 59.019 72.889 77.075 56.18
CLEAR: exp349-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 70.511 78.727 70.656 77.004 92.384 49.398 36.145 14.458 54.13 36621 10936 3019 69 41 30 12 416
MOT17-05-FRCNN 47.723 79.704 48.518 52.046 93.652 21.805 58.647 19.549 37.16 3600 3317 244 55 29 78 26 185
MOT17-09-FRCNN 71.023 84.514 71.437 76.939 93.326 53.846 46.154 0 59.108 4097 1228 293 22 14 12 0 79
MOT17-10-FRCNN 64.351 77.64 64.756 68.323 95.038 33.333 56.14 10.526 49.074 8772 4067 458 52 19 32 6 607
MOT17-11-FRCNN 65.526 85.418 65.759 80.617 84.438 48 38.667 13.333 53.77 7607 1829 1402 22 36 29 10 108
MOT17-13-FRCNN 54.278 77.33 54.604 58.444 93.835 31.818 39.091 29.091 41.028 6804 4838 447 38 35 43 32 256
COMBINED 65.496 79.602 65.771 72.027 92.008 35.95 46.281 17.769 50.804 67501 26215 5863 258 174 224 86 1651
Identity: exp349-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-FRCNN 74.865 68.633 82.341 32640 14917 7000
MOT17-05-FRCNN 56.129 43.661 78.564 3020 3897 824
MOT17-09-FRCNN 68.183 62.197 75.444 3312 2013 1078
MOT17-10-FRCNN 65.295 56.118 78.061 7205 5634 2025
MOT17-11-FRCNN 73.201 71.545 74.936 6751 2685 2258
MOT17-13-FRCNN 64.362 52.225 83.851 6080 5562 1171
COMBINED 70.634 62.965 80.432 59008 34708 14356
Count: exp349-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-FRCNN 39640 47557 135 83
MOT17-05-FRCNN 3844 6917 127 133
MOT17-09-FRCNN 4390 5325 44 26
MOT17-10-FRCNN 9230 12839 96 57
MOT17-11-FRCNN 9009 9436 154 75
MOT17-13-FRCNN 7251 11642 106 110
COMBINED 73364 93716 662 484
MOTA is a bit higher, but IDF1 goes down quite a lot. Not sure why this is the case as the ReID model used for both BoTSORT and StrongSORT is the same. Should a matter of tuning but don't have too much time for this now
You've already tried BoT-SORT 😂. And as mentioned here BoT-SORT requires a lot of computation as well. So are we dropping the plan??
You've already tried BoT-SORT :joy:
I tried it just when it came out, at that point in time it was not real-time with the provided code. Recently they added a new camera motion compensation method based on opticalflow which partially fixes the slow execution time. Another weak point was the provided ReID ResNet50 model (~100MB) which was too computationally heavy, so the fact that you added the multibackend for ReID is great. With all this fixes it is running almost as fast as StrongSORT. What is left is to find a configuration for it that leads to better results than StrongSORT :smile:
So are we dropping the plan??
Not at all. Given that MOTA already gets quite a big boost (+1.563) it should be feasible to fix the boost for IDF1 as well :smile:
So are we dropping the plan??
Not at all. Given that MOTA already gets quite a big boost (+1.563) it should be feasible to fix the boost for IDF1 as well 😄
Okay then, can you suggest some techniques to improve IDF1 so I can look into it. I mean you have worked a lot with trackers compared to me, so yaa.
I am checking if feature normalization can be the issue here. I would recommend that you check the ReID feature handling in StrongSORT to try to find inconsistencies between both.
I am checking if feature normalization can be the issue here. I would recommend that you check the ReID feature handling in StrongSORT to try to find inconsistencies between both.
Okay, will do that and get back to you.
Nope, normalization is not the issue here. It is handled in the track itself
https://github.com/NirAharon/BoT-SORT/blob/98173060469b656ae8653acab97b0e37887d6b1a/tracker/bot_sort.py#L35
Nope, normalization is not the issue here. It is handled in the track itself
https://github.com/NirAharon/BoT-SORT/blob/98173060469b656ae8653acab97b0e37887d6b1a/tracker/bot_sort.py#L35
Okay I'll check on my side as well and get back to you.
I found something interesting:
https://github.com/NirAharon/BoT-SORT/blob/98173060469b656ae8653acab97b0e37887d6b1a/tracker/matching.py#L160
The activated fuse is a simpler one than the one in the link above, which is closer to the one used in StrongSORT
I am quite confident that this is one of the reason for the lower IDF1 as the motion is ignored when building the cost matrix
I am quite confident that this is one of the reason for the lower IDF1 as the motion is ignored when building the cost matrix
Okay will look into it.
We are considering combined IDF1 on MOT17, right? If you checkout with this FairMOT tracker the fuse_motion function, the functionality is the same. Could we work on the Kalman filter side?
If you checkout with this FairMOT tracker the fuse_motion function, the functionality is the same.
Sure, but it is not activated. However I already tried it here:
https://github.com/NirAharon/BoT-SORT
I get much worse results with fuse_motion
Could we work on the Kalman filter side?
Given that the MOTA metric (emphasis on detections) is already good but that IDF1 (emphasis on classification) is low I don't think that the Kalman Filter is to blame here. I think it is the way the information is used to build the cost function.
This may lead to a good set of hparams:
git pull
git checkout hps
pip install -r requirements.txt
python3 val.py --tracking-method botsort --benchmark MOT17 --evolve --n-trials 1000
I will run a parameter evolution for this after the one i am running for StrongSORT
I will run a parameter evolution for this after the one i am running for StrongSORT
Sorry for the delay, bit busy with some other tasks.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}