About MPPNet "Train the RPN model for MPPNet (centerpoint_4frames is employed in the paper)"

[xpyqiubai]

Thanks the author share code about MPPNet. But based one the author readme, First I generated the new datasets about waymo, I want verify whether the dataset is true. This is the number and size of my generated files Folders waymo_processed_data_v0_5_0 contains 1000 Folders Folders waymo_processed_data_v0_5_0_gt_database_train_sampled_1 contains 2361803 items Folders waymo_processed_data_v0_5_0_gt_database_train_sampled_1_multiframe_-4_to_0 335917 File waymo_processed_data_v0_5_0_gt_database_train_sampled_1_global.npy 15.4 GB SIZE File waymo_processed_data_v0_5_0_gt_database_train_sampled_1_multiframe_-4_to_0_global.npy 85.3 GB SIZE File waymo_processed_data_v0_5_0_infos_train.pkl 3.2GB SIZE File waymo_processed_data_v0_5_0_infos_val.pkl 807.6MB SIZE File waymo_processed_data_v0_5_0_waymo_dbinfos_train_sampled_1.pkl 880.3MB SIZE File waymo_processed_data_v0_5_0_waymo_dbinfos_train_sampled_1_multiframe_-4_to_0.pkl 1.2GB SIZE

Second I based on the sh bash scripts/dist_train.sh ${NUM_GPUS} --cfg_file cfgs/waymo_models/centerpoint_4frames.yaml train the model, but I only get a poor performance, as shown below 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/AP: 0.7173 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/APH: 0.7116 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/APL: 0.7173 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/AP: 0.6394 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/APH: 0.6343 2022-09-16 10:40 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/APL: 0.6394 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/AP: 0.7503 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/APH: 0.7178 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/APL: 0.7503 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/AP: 0.6748 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/APH: 0.6447 2022-09-16 10:40 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/APL: 0.6748 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/AP: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/APH: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/APL: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/AP: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/APH: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/APL: 0.0000 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/AP: 0.6557 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/APH: 0.6453 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/APL: 0.6557 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/AP: 0.6336 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/APH: 0.6235 2022-09-16 10:40 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/APL: 0.6336 2022-09-16 10:40 This is a large performance gap from what the author shared

Model	Vec_L1	Vec_L2	Ped_L1	Ped_L2	Cyc_L1	Cyc_L2
centerpoint_4frames	76.71/76.17	69.13/68.63	78.88/75.55	71.73/68.61	73.73/72.96	71.63/70.89

Model Vec_L1 Vec_L2 Ped_L1 Ped_L2 Cyc_L1 Cyc_L2 centerpoint_4frames 76.71/76.17 69.13/68.63 78.88/75.55 71.73/68.61 73.73/72.96 71.63/70.89

I want to know, whether have some about config
https://github.com/open-mmlab/OpenPCDet/blob/master/tools/cfgs/waymo_models/centerpoint_4frames.yaml I notice that this config only use the SAMPLED_INTERVAL: { 'train': 5, 'test': 1 } It does not use all train data 'train': 1 Many thanks to the author for opening the source code and looking forward to the author's answer to help me to be able to successfully reproduce the performance in the MPPNet text.

[xpyqiubai]

I only get the the performance as shown below

方法	Vehicle L1	Vehicle L2	Ped L1	Ped L2	Cyclist L1	Cyclist L2
centerpoint 4帧 原文	76.71/76.17	69.13/68.63	78.88/75.55	71.73/68.61	73.73/72.96	71.63/70.89
centerpoint 4帧 Openpcdet our	71.74/71.17	63.95/63.44	75.04/71.79	67.49/64.47	65.54/64.50	63.32/62.31

[Cedarch]

Thanks for your report. You should change the config from SAMPLED_INTERVAL: { 'train': 5, 'test': 1 } to SAMPLED_INTERVAL: { 'train': 1, 'test': 1 } . Our paper use all datasets to train centerpoint and mppnet. We will change the default config to { 'train': 1, 'test': 1 } .

[xpyqiubai]

Thanks for your report. You should change the config from SAMPLED_INTERVAL: { 'train': 5, 'test': 1 } to SAMPLED_INTERVAL: { 'train': 1, 'test': 1 } . Our paper use all datasets to train centerpoint and mppnet. We will change the default config to { 'train': 1, 'test': 1 } .

Thank you I will retrain the model.

[hu-zheng]

@Cedarch Thanks for your work. Could you provide a trained centerpoint_4frames ckpt? Because the training time of centerpoint_4frames using all datasets is too long, I focus on the second stage of MPPNet training and want to reproduce the results faster.

[xpyqiubai]

@Cedarch Thanks for your work. Could you provide a trained centerpoint_4frames ckpt? Because the training time of centerpoint_4frames using all datasets is too long, I focus on the second stage of MPPNet training and want to reproduce the results faster.

Have you reproduced the results of the first stage yet? I train the centerpoint_4frames based { 'train': 1, 'test': 1 } for 100 hours. But I have not get the same performance with paper.

[xpyqiubai]

@Cedarch Thanks the author share code about MPPNet. Based on suggest about SAMPLED_INTERVAL: { 'train': 1, 'test': 1 } . I retrain the model. But I only get the performance as shown below.

方法	Vehicle L1	Vehicle L2	Ped L1	Ped L2	Cyclist L1	Cyclist L2
centerpoint 4帧 原文	76.71/76.17	69.13/68.63	78.88/75.55	71.73/68.61	73.73/72.96	71.63/70.89
centerpoint 4帧 1/5 our	71.74/71.17	63.95/63.44	75.04/71.79	67.49/64.47	65.54/64.50	63.32/62.31
centerpoint 4帧 our	74.10/73.54	66.46/65.96	76.83/73.34	69.51/66.28	71.20/70.17	68.95/67.95

This is still a large gap from the performance in the paper. Are there still parameters in the configuration file that need to be modified? Do you share your configuration files and logs of model training? To help me to reproduce the performance in the paper smoothly. Thank you very much and I look forward to your reply.

[hu-zheng]

@Cedarch Thanks for your work. Could you provide a trained centerpoint_4frames ckpt? Because the training time of centerpoint_4frames using all datasets is too long, I focus on the second stage of MPPNet training and want to reproduce the results faster.

Have you reproduced the results of the first stage yet? I train the centerpoint_4frames based { 'train': 1, 'test': 1 } for 100 hours. But I have not get the same performance with paper.

I haven't reproduced the results based SAMPLED_INTERVAL: { 'train': 1, 'test': 1 }. I trained a centerpoint_4frames based SAMPLED_INTERVAL: { 'train': 5, 'test': 1 } and the result is as follows

[xpyqiubai]

@Cedarch Thanks for your work. Could you provide a trained centerpoint_4frames ckpt? Because the training time of centerpoint_4frames using all datasets is too long, I focus on the second stage of MPPNet training and want to reproduce the results faster.

Have you reproduced the results of the first stage yet? I train the centerpoint_4frames based { 'train': 1, 'test': 1 } for 100 hours. But I have not get the same performance with paper.

I haven't reproduced the results based SAMPLED_INTERVAL: { 'train': 1, 'test': 1 }. I trained a centerpoint_4frames based SAMPLED_INTERVAL: { 'train': 5, 'test': 1 } and the result is as follows

I also have the same performance of "centerpoint_4frames based SAMPLED_INTERVAL: { 'train': 5, 'test': 1 }". The performance as shown below.

方法	Vehicle L1	Vehicle L2	Ped L1	Ped L2	Cyclist L1	Cyclist L2
centerpoint 4帧 1/5 our new	75.69/75.14	67.94/67.44	77.30/74.05	69.97/66.94	70.75/69.82	68.50/67.60

[Cedarch]

Sorry for late reply. Can you send me the training log and ckpt of centerpoint_4frame to me ? @xpyqiubai, @hu-zheng. My email is [email protected]

[hutao568]

@xpyqiubai Hello, can you give me a pre-trained model of centerpoint_4frames, it will take me 7 days to train with my own 4-card A30, which is too long.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.