Tracker skips a few identities

[sampreets3]

Hello @abewley

I am using the SORT tracker for tracking trees in a vineyard, and from what I observe, the tracker is skipping a couple of IDs during assignment. In the video file below, the tracker is skipping the ID 11, 14, 16, 17, and so on. I cannot figure out why this happens.

I need to find a fix to this and I am not seeing any better results even after setting the parameters. My current parameter settings are as follows:

"max-age"   : "10",
"min-hits"  : "10",
"iou-thresh": "0.1"

Is there any way I can re-assign the IDs or make it more robust? I would really like to have this being done at the tracker end, rather than having to re-assign the IDs at the application end.

Thanks for the help!

[varungupta31]

@sampreets3 Hey, can you please explain how your are visualizing tracker results on custom video?

[sampreets3]

Hey @varungupta31, I don't remember the exact details at the moment as I worked on this quite some time ago, but if I remember correctly, The output of the Sort.update() function is of the format [x1, x2, y1, y2, track_id]. You can also see the working example in the main section of the sort.py file.

Sorry for not being able to help a lot with this at the moment, I'll try and whip up a working example for you if I get the time. I hope you get the general idea of how I implemented it.

[varungupta31]

@sampreets3 The first 5 lines of the output that I got were: (it starts from 2, I don't know why - maybe because I set the max_age=1, min_hits=0)

2,1,1291.00,957.00,55.28,109.84,1,-1,-1,-1
3,1,1291.02,956.95,55.36,109.81,1,-1,-1,-1
4,1,1291.02,956.96,55.46,109.91,1,-1,-1,-1
5,1,1290.99,956.95,55.31,109.60,1,-1,-1,-1
6,1,1290.94,956.95,54.99,109.06,1,-1,-1,-1

I think - first number is the frame, second is the track ID, remaining are bounding box (Don't know what - top, left, center, etc.), and the rest I dont need. Am I correct? And do you know what the bounding box coordinated are? (The last 2 seem to be the width and the height)

No worries, thanks for the prompt help :)

[sampreets3]

Hi @varungupta31 you are right, the first element is the frame id, elements 3 - 6 are bounding box coordinates in the format (x, y, w, h), the second element might be the tracked id. I don't remember at the moment what the last four elements correspond to, but I remember having looked up the format somewhere - can't place it right now.

Could you provide me a code sample of how you used the tracker, if it is possible? I just want to refresh my memory on how to do that :D

Also, pay attention to the fact that when you draw the bounding boxes with these coordinates, you need to cast them as integers if you are using OpenCV.

I'll try and dig up some more details on the format if I can.

[varungupta31]

Hi @sampreets3

Could you provide me a code sample of how you used the tracker

So I'm working with JSON files that had the detection coordinates from my detector (YOLOv4). I basically took those detections, scaled them as per my video, and created a DET (.txt) file for the input for the SORT-Tracker, in the format: frame number, -1, bounding box coordinates, confidence, -1, -1, -1 for each frame in my detection JSON.

After this, I had to modify the sort.py code so that the command line prompt works for custom DET files (original code expected the file to be named 'det.txt' exactly). After some tweaks, I ran the tracker using

python sort.py --phase <path to DET file subdir> --max_age 1 --min_hits 0 this generated the tracker output.

I ended up creating and modifying a script myself to write the bounding box coordinates on my video using openCV I was searching for a way to modify the sort.py in such a way that --display flag works for custom video as well, but nevetheless, my goal is achieved :P

[andreaceruti]

@sampreets3 Hi! Since I have the same problem could you tell me how did you solve this issue? I am tracking grape bunches and some IDs are missing, can't understand how to fix this problem

[sampreets3]

Hey @andreaceruti if you take a look at the Sort class, you will see that it uses the bboxes itself and not the features detected within the bboxes to track an object. This proves to be a little tricky when objects are in close proximity to each other, like grape bunches.

Afaik Deep Sort uses some sort of metric to associate features as well as the bboxes. I ended up using the Deep Sort tracker, which gave slightly better results.

Also, if you want to keep using the Sort tracker, I would advise you to play around with the min_age and max_hits to provide a better context for your tracker.

Hope this helps! 😃

[andreaceruti]

Thank for your quick reply @sampreets3! Unfortunately my camera does not face in a parallel way the vineyard (as I can see in your video) and in fact if you see in the right side some IDs are missing as in your problem.

This "IDs skipping problem" does not change even if I play a bit with the parameters since this problem is probably due to the tracklets that are created and then dropped by the algorithm. I do not have investigated a lot the code and so I can not tell how to reassign the IDs. Anyway some that can be done as you said is to reassign the IDs manually once the tracking is finished

[sampreets3]

Hi @andreaceruti it seems the quicker option right now for you would be to manually reassign the tracks once the tracking is finished. Some tips on that front : for my application, I collected everything into a custom JSON and then aligned the tracked element and its corresponding bounding box with the frame number. The gross idea was that if there has been an IDx for n number of frames, and if the bboxes have been slowly evolving in a direction (can be determined by checking the bbox co-ordinates), then we can conclude that the tracklet has come and gone. If however, the track is lost and a new ID is found somewhere near the center of the image, we can assume that this is a miscalculatation. I don't know if this would help your case, but it kind of improved the results for me - not by much though.

All the best 😃

[andreaceruti]

Thank you @sampreets3 for the tips and the brainstorming! All the best to you too!