About the results in ps2.0

[Vipermdl]

trafficstars

Hi, Teoge!

Thanks for your contribution about parking slot, I'm appreciated for it. But I have an question about your result:

The paper said the result of DMPR has recalls of 99.37% and precisions of 99.42%, But I test you released pre-trained weights only achieved the recalls of 49.6% and precisions of 99.5%, the evaluation in code I used:

================================================================== precisions, recalls = util.calc_precision_recall(ground_truths_list, predictions_list, match_slots) print(np.mean(precisions), np.mean(recalls))

So, What can I do to re-implement the result about your paper said?

[Teoge]

You misunderstood the output results of calc_precision_recall. This function adjusts thresholds automatically and produce the curve of precision-recall. You can plot the curve and pick any point of them as a precision-recall result under a specific threshold. As far as I see, calculating the means of these precisions and recalls does not produce any meaningful results. If you still have trouble understanding this, you can read util.py for how I calculate precisions and recalls.

[Vipermdl]

Thanks for your answer. You set the threshold of 0.000277778, which means 10 pixel in 600*600 image just like your supervisor's paper said. And then, we used the function of calc_precision_recall to get a list of precisions, and recalls, which like AP of object detection. But I still want to know how can I re-implement the result of recalls 99.37% and precisions 99.42%. Looking forward to your reply！

[Teoge]

The threshold I was talking about is the confidence threshold. Anyway, here are the specific steps to reproduce the results.

1. Download dataset and weights.
2. Start visdom server with command python -m visdom.server.
3. Run ps_evaluate.py with visdom enabled. python ps_evaluate.py --label_directory $LABEL_DIRECTORY --image_directory $IMAGE_DIRECTORY --detector_weights $DETECTOR_WEIGHTS --enable_visdom
4. Go to http://localhost:8097. The precision-recall curve should be ploted as the figure below. The x-axis represents the recall rate and the y-axis represents the precision rate. As shown in the figure, we have a precision rate of 99.42% and a recall rate of 99.37%.