SST icon indicating copy to clipboard operation
SST copied to clipboard

meaning of provided result files

Open yrt1993 opened this issue 2 years ago • 2 comments

Hi, I want to know the detailed meaning of the result file in the resources provides in the link: https://github.com/tusen-ai/SST/blob/main/docs/CTRL_instructions.md#results

specifically, how do you get these files? i.e., their difference? fsd_base_ped_val.bin fsd_base_ped_val_forward10.bin ped_result.bin ped_result_forward10_no_empty.bin

yrt1993 avatar Sep 05 '23 07:09 yrt1993

fsd_base_ped_val.bin: tracking/detection results of base detector fsd_base_ped_val_forward10.bin: tracking/detection results of the base detector with filling missing boxes in forward extension. ped_result.bin: CTRL baseline results on Pedestrian without bidirectional extension ped_result_forward10_no_empty.bin: CTRL results on Pedestrian with forward extension and the empty boxes are removed. forward10 means filling missing boxes in forward extension, filling at most 10 consecutively missing timestamps.

Abyssaledge avatar Sep 06 '23 03:09 Abyssaledge

I tried to use the four files to compute waymo metrics. The following is what I get. It's really large just using fsd_base_ped_val.bin. Are you sure it's just the results of base detector?

And what's the detector?

fsd_base_ped_val.bin OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.863252] [mAPH 0.838515] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.798981] [mAPH 0.774175] fsd_base_ped_val_forward10.bin OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.870338] [mAPH 0.845087] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.808826] [mAPH 0.783306] ped_result.bin OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.872514] [mAPH 0.845624] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.809663] [mAPH 0.782687] ped_result_forward10_no_empty.bin OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.887341] [mAPH 0.859652] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.828941] [mAPH 0.800911]

yrt1993 avatar Sep 06 '23 05:09 yrt1993