RTM3D
RTM3D copied to clipboard
Banconxuan
Results interpretations
Hi,
first of all let me thank you for this repo. I would like to ask you about the interpretation of results. When I run the evaluation using your DLA-34 model, I get this:
Car [email protected], 0.70, 0.70: bbox AP:90.84, 89.72, 80.89 bev AP:24.48, 18.83, 17.77 3d AP:17.88, 12.67, 12.00 aos AP:90.31, 88.66, 79.69 Car [email protected], 0.70, 0.70: bbox AP:96.96, 91.32, 83.77 bev AP:23.28, 17.16, 14.86 3d AP:15.95, 11.41, 9.55 aos AP:96.34, 90.18, 82.39 Car [email protected], 0.50, 0.50: bbox AP:90.84, 89.72, 80.89 bev AP:61.77, 46.04, 43.18 3d AP:56.54, 43.31, 37.07 aos AP:90.31, 88.66, 79.69 Car [email protected], 0.50, 0.50: bbox AP:96.96, 91.32, 83.77 bev AP:61.44, 45.71, 39.51 3d AP:55.12, 41.15, 35.51 aos AP:96.34, 90.18, 82.39 Pedestrian [email protected], 0.50, 0.50: bbox AP:69.26, 60.87, 52.07 bev AP:12.50, 11.99, 11.42 3d AP:12.28, 11.33, 10.71 aos AP:61.45, 53.35, 45.55 Pedestrian [email protected], 0.50, 0.50: bbox AP:67.03, 57.51, 47.95 bev AP:6.74, 5.34, 4.33 3d AP:5.73, 4.27, 3.46 aos AP:59.17, 50.00, 41.50 Pedestrian [email protected], 0.25, 0.25: bbox AP:69.26, 60.87, 52.07 bev AP:30.74, 25.89, 20.53 3d AP:30.47, 25.37, 20.38 aos AP:61.45, 53.35, 45.55 Pedestrian [email protected], 0.25, 0.25: bbox AP:67.03, 57.51, 47.95 bev AP:25.65, 20.78, 17.36 3d AP:25.41, 20.47, 17.11 aos AP:59.17, 50.00, 41.50 Cyclist [email protected], 0.50, 0.50: bbox AP:73.84, 48.44, 47.94 bev AP:15.24, 11.49, 11.31 3d AP:10.21, 6.54, 6.46 aos AP:70.27, 45.87, 45.24 Cyclist [email protected], 0.50, 0.50: bbox AP:74.42, 46.35, 43.89 bev AP:8.87, 4.61, 4.40 3d AP:7.27, 3.70, 3.42 aos AP:70.55, 43.61, 41.22 Cyclist [email protected], 0.25, 0.25: bbox AP:73.84, 48.44, 47.94 bev AP:32.14, 19.41, 18.73 3d AP:29.18, 19.24, 18.56 aos AP:70.27, 45.87, 45.24 Cyclist [email protected], 0.25, 0.25: bbox AP:74.42, 46.35, 43.89 bev AP:28.15, 15.16, 13.49 3d AP:26.90, 14.91, 13.17 aos AP:70.55, 43.61, 41.22
Can you please tell me what results correspond to? They differ from the values that you present at this page.
Thank you very much in advance.
Thank you for your reply. I suppose that the reported results are for the standard AP (11 points) and not for the new AP_40 with 40 points, right?
Also, the training in main.py uses KM3D so it should be evaluated with faster.py?
And one more question: how does it come that there are two different results for each category? E.g., two results for Car AP - 0.7, 0.7, 0.7 setup and 0.7, 0.5, 0.5 setup. I guess that those might be different setups for easy/moderate/hard, but I don't understand why would I get different result for 0.7 (first column) in both tables, or why would the first rows (2D bounding boxes) be the same.
Thank you very much in advance.
Upon running faster.py, we do not see any results like the ones you posted above @vobecant. Only the time stats, are printed to the terminal. Are the stats you saw being printed to a separate file and if so where is that file located within the repo? We've looked through the code, but have yet to find such a file. Thanks
@sparro12, faster.py does just the detection.
You then need to run src/tools/kitti-object-eval-python/evaluate.py for the evaluation.