SFA3D icon indicating copy to clipboard operation
SFA3D copied to clipboard

Add the kitti evaluation code

Open CarkusL opened this issue 3 years ago • 6 comments

I add some code to do evaluation on kitti dataset. The evaluation code is from https://github.com/traveller59/kitti-object-eval-python.

Evaluation result

Car AP(Average Precision)@0.70, 0.70, 0.70:
bbox AP:96.57, 89.17, 89.41
bev  AP:97.52, 89.62, 89.78
3d   AP:88.09, 87.86, 88.09
aos  AP:60.28, 55.63, 55.04
Car AP(Average Precision)@0.70, 0.50, 0.50:
bbox AP:96.57, 89.17, 89.41
bev  AP:98.03, 89.94, 90.09
3d   AP:98.01, 89.94, 90.09
aos  AP:60.28, 55.63, 55.04
Pedestrian AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:65.38, 64.95, 63.96
bev  AP:68.14, 69.36, 65.41
3d   AP:66.44, 62.36, 62.77
aos  AP:32.13, 30.95, 30.21
Pedestrian AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:65.38, 64.95, 63.96
bev  AP:88.43, 88.61, 88.59
3d   AP:88.29, 88.45, 88.48
aos  AP:32.13, 30.95, 30.21
Cyclist AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:89.62, 87.64, 87.70
bev  AP:82.11, 75.41, 75.64
3d   AP:80.09, 74.31, 74.45
aos  AP:54.37, 52.01, 51.44
Cyclist AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:89.62, 87.64, 87.70
bev  AP:96.04, 88.67, 88.78
3d   AP:96.04, 88.67, 88.78
aos  AP:54.37, 52.01, 51.44

CarkusL avatar Aug 22 '21 11:08 CarkusL

Thanks for your PR

maudzung avatar Aug 25 '21 13:08 maudzung

I was using only lidar data and label data. So, I can't achieve evaluation because I don't have calibration and 2d coordinate data. Can you help me?

reddittoob avatar Apr 08 '22 06:04 reddittoob

Hello, can you please tell me how can I display the Precision and Recall values, I need them for my paper, thank you.

mouawiya avatar Sep 03 '22 15:09 mouawiya

Hello, can you please tell me how can I display the Precision and Recall values, I need them for my paper, thank you. You can use my branch https://github.com/CarkusL/SFA3D

CarkusL avatar Sep 04 '22 03:09 CarkusL

Hello, can you please tell me how can I display the Precision and Recall values, I need them for my paper, thank you. You can use my branch https://github.com/CarkusL/SFA3D I am using your branch, but it still only display the AP in the results, I want to display the Precision and the Recall values with the Average precision (AP)

mouawiya avatar Sep 04 '22 07:09 mouawiya

@maudzung Hello, I found that your ImageSets files are not following the conventions in KITTI 3D, which often use 3712 pointcloud bin as train-split and 3769 as val-split. The split file can refer from OpenPCDet repo https://github.com/open-mmlab/OpenPCDet/tree/master/data/kitti/ImageSets.

Xi-Gong avatar Mar 30 '24 12:03 Xi-Gong