Difference implementations between this repo and the faster_rcnn ones

[ksmdnl]

Thank you for the exhaustive resources that you put in here for the community to learn from!

I've been wondering what the difference between the implementation of mAP for pascal voc from this repository and the ones from https://github.com/longcw/faster_rcnn_pytorch.git

After spending sometime 'debugging' I came to the conclusion that both should be identical. However, when comparing the result, I get better mAP using the implementation from this repo (75.6) than using the other one (71.2).

I wonder if anyone have had similar problem or dilemma and could share their experiences?

[Flova]

Thank you!

Zudem hallo nach Braunschweig :D

I am currently a bit busy, so I am not able to dive too deep into the different implementations, but I noticed, that the implementation in the repo you linked has the use_07_metric setting, which is set here. I don't know the exact differences, but this might affect it. The last time I checked, this repos mAP nearly equaled the official YOLO mAP using the official weights on coco. I hope that this might help.

Also, how did you compare the mAPs, as this repo is for YOLO and the other one is for faster rcnn, the VOC dataset code is also not provided for this repo. Any chances some bug happened during the implementation? If you do some more research on this, try to make a minimal reproduction, that is reproducable and post it here. :)

[ksmdnl]

VG zurück nach Hamburgo :D Thanks for the heads up. Enabling use_07_metric allows to calculate the mAP based of the precision v. recall curve using the 11 points interpolation. Not enabled, then the all points interpolation is selected. Indeed this gives different results.

In my case for the debugging I kept it off, thus the all points interpolation, on both setups. The way I compare them was actually quite straightforward, as the mAP concerns only to the precision/recall curve to my understanding. I took the final detections and the ground truth labels and from there calculated the mAP, one using this repo (which is based on the darknet implementation), another using the one from voc_eval. At glance (couple hours of debugging) they should do the same thing or might be that I'm biased and wanted to believe they are equivalent. I could post my detection results and the ground truth from a couple samples here.

The only difference is, the latter loads the ground truth separately instead of tagging the samples directly like this or darknet implementation.