Different score_threshold result in different mAP

[yd8534976]

In lmfyolo config file, we use default score_threshold = 0.05. If we change the score_threshold to another value. We may get different mAP.

Here, I try to change the score_threshold to 0. mAP will get larger. Results as below:

Maybe we should fix the score_threshold to a smaller value when training to calculate mAP.

[patrick-su]

@yd8534976 @masato0412 OK, after given some thoughts, I think this is what happened (assuming the mAP calculation is correct)

1. confidence score threshold lower
2. a lot of low-confidence boxes all count as prediction
3. since there are a lot of "random boxes", precision goes to 0%, and recall goes to 100%
4. however, for mAP, this only mean we have more sampling points on the right side of precision_x_recall curve, and have better approximation of real curve there.
5. area under curve (mAP) increases slightly due to better approximation

reference: https://github.com/rafaelpadilla/Object-Detection-Metrics

[joelN123]

yes, it looks like there are some low score boxes that actually were correct? Maybe it is good to lower that threshold... Another point is that maybe the non-maximum suppression will have to do more work because there are more boxes to compare (NMS is used after excluding low scores). Possibly it affects the speed of the post process.