Why not use argmax of scores for label in suppress

[tayden]

trafficstars

In the FasterRCNN.py module's _suppress function, All BG class scores are ignored and the FG box scores are kept as long as they are > 0.05. These then go into a class-wise NMS function.

Wouldn't it be better to first label all the boxes using the argmax of the probability scores, and then discard the BG boxes? It doesn't make sense to me to keep a box because it has a FG class confidence score of 0.1 if the BG class has a score of 0.9.