augmented NuScene as inputs to petrv2_BEVseg.py - inference output 3d boxes shifted and scaled

[mdfaheem786]

Hi,

i am using this config https://github.com/megvii-research/PETR/blob/main/projects/configs/petrv2/petrv2_BEVseg.py and its respective model.

But I changed the test pipeline, with scale(twice-640x1600) and and rotation(degress-5), with this in the flow its augmenting the input images and lidar2imgs matrices too.

For rotation of 5 degrees at least output is reasonable, but for scaled(double) output is going weird, as attached below. please correct me if I am wrong. Attached are the samples

[yingfei1016]

Hi， Can you provide the test pipeline and augmentation parameters (https://github.com/megvii-research/PETR/blob/main/projects/configs/petrv2/petrv2_vovnet_gridmask_p4_800x320.py#L150).

[mdfaheem786]

Hi,

thanks for the quick reply :)

for scaled: ida_aug_conf = { "resize_lim": (0.47, 0.625), # "final_dim": (320, 800), "final_dim": (640, 1600), "bot_pct_lim": (0.0, 0.0), "rot_lim": (0.0, 0.0), "H": 900, "W": 1600, "rand_flip": True, }

for rot5 degree ida_aug_conf = { "resize_lim": (0.47, 0.625), # "final_dim": (320, 800), "final_dim": (640, 1600), "bot_pct_lim": (0.0, 0.0), "rot_lim": 5, "H": 900, "W": 1600, "rand_flip": True, }

for rotation i have use this value directly from config, rather initialize to zero (https://github.com/megvii-research/PETR/blob/main/projects/mmdet3d_plugin/datasets/pipelines/transform_3d.py#L464)

thank you :)

[yingfei1016]

Hi，

The "resize_lim" must be scaled with "final_dim". When use (640, 1600), the "resize_lim" should be (0.94, 1.25).

[mdfaheem786]

Hi @yingfei1016 ,

thanks for the reply :)

i am doing inference, so in this flow "resize_lim" is not used in else part as shown below screenshot

Only getting used in training mode ie., if part. thank you :)

[yingfei1016]

Hi， You are right, "resize_lim" is not used during inference. I re-checked the test pipeline and found no problem. By default, the test pipeline does not read gt boxes. In order to visualize during the test process, some test code needs to be modified. So I think you can see if there's a problem in the training process. If the training is normal, it may be that some bugs during the inference.

[mdfaheem786]

Hi @yingfei1016 ,

thanks for quick reply.

I am using pretrain trained model (from GitHub link), just checking whether inference will be good if I change the resolutions of input.

So that any other dataset also like waymo also work with this pretrained model, but unfortunately it didn't work. (https://github.com/megvii-research/PETR/issues/102) uploaded the results too.

Then tried with nuScenes with augmentation with doubling the size as explained above.

thank you :)