pandaset-devkit icon indicating copy to clipboard operation
pandaset-devkit copied to clipboard

How to evaluating results?

Open Leozyc-waseda opened this issue 3 years ago • 7 comments

Hi,there I am trying to test kitti trained models like pv_rcnn on pandaset. it looks like there is no standard evaluation metric yet.

can you tell me how to do it?

Leozyc-waseda avatar Feb 09 '22 13:02 Leozyc-waseda

Hi,there I am trying to test kitti trained models like pv_rcnn on pandaset. it looks like there is no standard evaluation metric yet.

can you tell me how to do it?

I also have this question,how you solve it?

Joeless avatar Mar 16 '22 02:03 Joeless

According to the pandaset paper, they are using kitti evaluation metric.

But I don't know how to convert to Kitti format, and the paper doesn't give a tutorial, I'm still stuck in this place, waiting for the official evaluation metric.

By the way, after I converted Pandaset's data into Bin format and tested, there is no result...

Leozyc-waseda avatar Mar 16 '22 10:03 Leozyc-waseda

Do you have wechat?, I am stuck int this place too I want to talk with you~ my wechat: MegaloAiming

Joeless avatar Mar 18 '22 13:03 Joeless

According to the pandaset paper, they are using kitti evaluation metric.

But I don't know how to convert to Kitti format, and the paper doesn't give a tutorial, I'm still stuck in this place, waiting for the official evaluation metric.

By the way, after I converted Pandaset's data into Bin format and tested, there is no result...

Do you get the AP? I reappear pandaset on 3D model ,but I get the higher AP in car ,the lower AP in Ped&cyclist

Joeless avatar Mar 23 '22 07:03 Joeless

@Joeless Sorry for the late reply, you can contact me by email. I tried converting pandas to kitti's bin format, but I didn't get any results (AP=0), and then I gave up Debug. BTW, can you share your approach?

Leozyc-waseda avatar Mar 23 '22 07:03 Leozyc-waseda

@Joeless Sorry for the late reply, you can contact me by email. I tried converting pandas to kitti's bin format, but I didn't get any results (AP=0), and then I gave up Debug. BTW, can you share your approach?

I just transfrom to kitti's format,using kitti eval

Joeless avatar Mar 25 '22 08:03 Joeless

Can you share your code? :)

Leozyc-waseda avatar Mar 25 '22 09:03 Leozyc-waseda