Bad results on the detectron2 example

[sadimoodi]

Hello, I have tired your detectron2 example on my video and i got very bad results, tracking is lost when even a minor occlusion happens, tried different detectron2 models but the results were consistently bad. any parameters to optimize?

[aguscas]

Hello sadimoodi! I don't know what may be causing your specific problem, but take into account that the distance function used in the detectron2 demo doesn't take into account the size of the objects or the resolution of your video, since it is just the unscaled distance between the centers. If your video has really high resolution, then the distance between objects can be large even for objects that appear to be close in the video. You can try re scaling the distance_function to take into account you video resolution and change the distance_threshold accordingly, or using a completely different distance_function.

If you think that your objects are getting deleted too fast, you can increase the hit_counter_max parameter so that the objects live longer without detections.

On another note, I do not know the quality of your detections. In case you do not have consistent detections (for example, if you are missing detections during really minor occlusions), try lowering the detection threshold of your detector and try lowering the detection_threshold argument you are currently using in the tracker (if you are using any non None value).

If you want to share your video with us so we can see it, it might be helpful in order to give you better advices.

[nikky4D]

@aguscas How do you take into account the size of the objects or the resolution of the video in the distance function? is it just dividing the norm by the video resolution? What would be the best strategy to incorporate this into the distance function and tracker?

account the size of the objects or the resolution of your video, since it is just the unscaled distance between the centers. If your video has really high resolution, then the distance between objects can be large even for objects that appear to be close in the video. You can try re scaling the distance_function to take into account you video resolution and change the distance_threshold accordingly

[aguscas]

Sorry for my late reply. Yes, what you said will be enough to take into account the resolution of the video, just redefine the distance function by returning the euclidean distance divided by the vertical resolution. Remember that the distance_threshold should be modified accordingly, because in that case a distance of 1 means that the euclidean distance between the object and the detection corresponds to 1 vertical resolution. You can play with different values for the threshold, I would say something between 0.1 and 0.3 would be reasonable for this example, but feel free to play with different values.

Depending on your use case, it could be more reasonable to divide the euclidean distance between the detection diameter instead of the resolution, so that a distance of 1 means that the euclidean distance between the object and the detection is comparable to the size of the detection. This can be useful if for example some objects might be too close to the camera (creating large detections) and others might be far away from it (creating really small detections).