output from, show_result_meshlab() with custom trained model doesn't show bounding box

[ammaryasirnaich]

I have trained a custom model along with validation and evaluation output. But when I tried to test the model with the below code

model = init_model(config_file, checkpoint_file, device='cuda:0')
pcd = 'kitti_000008.bin'
result, data = inference_detector(model, pcd)
out_dir = './'
show_result_meshlab(data, result, out_dir)

it doesn't seem to predict any bounding boxes. However, I also tried pre-trained SECOND and POINTPILLER, which do shows predicted bounding boxes when .obj output files are imported in cloudcomapare tool.

Is there anything that I am missing in-between?

[Tai-Wang]

You may add breakpoints in your codes and have a look at if there is any boxes output. Maybe your model is not trained well, or there is some inconsistency between your coordinate system and the demo file.

[ZCMax]

Did you train your custom model on kitti datasets? or on your custom datasets? If latter, mind the difference between your custom datasets and kitti datasets.

[ammaryasirnaich]

@Tai-Wang, @ZCMax thanks for your suggestions. I have actually used the configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py as a base model and i am using KITTI dataset

I think the evaluation results shown below seem to have trained to some extent. ----------- AP11 Results ------------

Car [email protected], 0.70, 0.70: bbox AP11:96.0997, 89.6783, 88.7846 bev AP11:90.0273, 87.9401, 84.7328 3d AP11:87.6342, 77.2437, 75.4011 aos AP11:95.88, 89.21, 88.01 Car [email protected], 0.50, 0.50: bbox AP11:96.0997, 89.6783, 88.7846 bev AP11:96.3633, 90.0843, 89.5578 3d AP11:96.2758, 90.0177, 89.3870 aos AP11:95.88, 89.21, 88.01

----------- AP40 Results ------------

Car [email protected], 0.70, 0.70: bbox AP40:97.7808, 92.3863, 91.4330 bev AP40:94.3739, 88.7645, 86.1023 3d AP40:89.4426, 78.5666, 75.6616 aos AP40:97.57, 91.86, 90.58 Car [email protected], 0.50, 0.50: bbox AP40:97.7808, 92.3863, 91.4330 bev AP40:98.0175, 95.0451, 94.4686 3d AP40:97.9622, 94.8429, 92.2996 aos AP40:97.57, 91.86, 90.58

[ammaryasirnaich]

I have tried to retrain the SECOND(voxelnet) model using ../configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py configuration file. After that, I used the generated checkpoint file epcoh_40.pth to do the prediction and used show_result_mesh() for visualization. But it shows no bounding box's on point clouds. I have added a breakpoint in mmdetection3d/mmdet3d/apis/inference.py line no 339 which tends to show no pred_bboxes.

 if 'pts_bbox' in result[0].keys():
        pred_bboxes = result[0]['pts_bbox']['boxes_3d'].tensor.numpy()
        pred_scores = result[0]['pts_bbox']['scores_3d'].numpy()

On the other hand, if I used the available pre-trained checkpoint to initialize the model and it does show the bounding box. Don't know what is causing this discrepancy.