panoramic-object-detection icon indicating copy to clipboard operation
panoramic-object-detection copied to clipboard

Published 20 hours ago • verified publisher icon gdlg

Benchmark Leaderboard

Open plaut opened this issue 5 years ago • 4 comments

Do you maintain a leaderboard for 3D object detection on your benchmark synthetic dataset?

Where can I find instructions for submitting my results?

plaut avatar Dec 22 '19 11:12 plaut

Hello, I haven’t had any interest yet in a leaderboard but I am happy to make one if that’s of interest to you.

Please email your results on the test dataset (*.txt files, same format as the KITTI benchmark) to me and I will run the evaluation against the groundtruth and email back the results to you.

gdlg avatar Dec 22 '19 16:12 gdlg

Thank you. I have a few more questions about the dataset:

  1. In the validation set there are no DontCare areas, yet there are clearly visible cars that are not labeled at all (and they are larger than 25 pixels). Is this also true for the test set?

  2. It appears that the horizontal center line in the panorama images is not the spherical projection's horizon, it is shifted by 10 degrees. Can you confirm?

plaut avatar Dec 26 '19 15:12 plaut

  1. This is a problem with the dataset. The static vehicles in the simulation were not captured in the list of vehicles (something that I thought that I had fixed). Thanks for spotting that. I can add DontCare areas but alas it is going to take some time to do so.

  2. Yes, that’s correct. The horizon is 88 pixels from the top. Numerically, I get 10.9°:

shift_pix = 300/2 - 88
angular_precision = 1024/pi
shift_rad = shift_pix / angular_precision
shift_deg = shift_rad * 180/pi

gdlg avatar Dec 27 '19 16:12 gdlg

If you can upload an updated dataset enough time before March 5, 2020 I would be happy to test on it and compare my results to your method.

plaut avatar Dec 29 '19 10:12 plaut