second.pytorch nuscenes trainval data generation dimension mismatch

Hi, I am running the create_data.py script to create the trainval database for training. I get the following error: ValueError: cannot reshape array of size 39424 into shape (5) for this trainval/sweeps/LIDAR_TOP/n015-2018-08-02-17-28-51+0800__LIDAR_TOP__1533202202399496.pcd.bin cloud file. I verified that the shape is indeed different and added a check in the get_sensor_data function to check points.size % 5 != 0 and ignore the files not satisfying the condition.

There is no issue in the mini-val set.

1 - Do others face this issue too? 2 - How did you resolve it? 3 - If we ignore the sweep files, does it cause other issues?

Oct 01 '19 21:10 rsarora

The file was probably corrupted during the download. See https://github.com/nutonomy/nuscenes-devkit/issues/233.

Oct 17 '19 02:10 leslieliang

I am getting the same error but when using the Lyft dataset which is based on the NuScenes format https://level5.lyft.com/dataset/

Oct 27 '19 16:10 jackvial

@rsarora Hi, I get the same error but may I know how do you know which cloud file it refer to? I only able to see the "ValueError: cannot reshape array of size 8192 into shape (5)", it never told me which cloud file it refer to. Thanks.

Oct 24 '20 05:10 AngYikHong