Issue Between Create Data and Training.

[chrisakatibs]

Hello,

My issue lies in the generation of data files that looked completed.

Kitti info train file is saved to /content/gdrive/MyDrive/voxel_data/kitti_infos_train.pkl Kitti info val file is saved to /content/gdrive/MyDrive/voxel_data/kitti_infos_val.pkl Kitti info trainval file is saved to /content/gdrive/MyDrive/voxel_data/kitti_infos_trainval.pkl Kitti info test file is saved to /content/gdrive/MyDrive/voxel_data/kitti_infos_test.pkl [100.0%][===================>][2.11it/s][01:19:45>00:00]
[100.0%][===================>][57.81it/s][02:51>00:00]
[100.0%][===================>][0.65it/s][56:13>00:01]
remain number of infos: 9996 [100.0%][===================>][8.81it/s][07:26:21>00:00]
load 249862 car database infos load 12990 bicycle database infos load 15167 pedestrian database infos load 14779 other_vehicle database infos load 6122 truck database infos load 4869 bus database infos load 654 motorcycle database infos

Now there are namespaces calling for a dbinfos file but it seems to be kitti_infos_train.pkl However when using these in the config I run into improper structures by the pickle dump? Perhaps my file structure was wrong? It would like to see a Dictionary to call .items() but sees a LIST of Dictionaries. I've tried to understand the structure and make short term solutions to see where it fails next but I am missing something.

Random error below from attempts at solving(printing out label info in question). This dict would be list given from pickle file dump[@ 0 index]

Please HELP!

Side note: using Lyft nuscenes dataset(but converted to kitti for both 3d and 2d operations), using Google Colab

{'image': {'image_idx': '805729458ae49dd0d72e7e428cdea67db775ceb2aa46e26ca9c5c0eb84823cae', 'image_path': 'training/image_2/805729458ae49dd0d72e7e428cdea67db775ceb2aa46e26ca9c5c0eb84823cae.png', 'image_shape': array([1080, 1920], dtype=int32)}, 'point_cloud': {'num_features': 4, 'velodyne_path': 'training/velodyne/805729458ae49dd0d72e7e428cdea67db775ceb2aa46e26ca9c5c0eb84823cae.bin'}, 'calib': {'P0': array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 1.]]), 'P1': array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 1.]]), 'P2': array([[1.10935803e+03, 0.00000000e+00, 9.60013076e+02, 0.00000000e+00], [0.00000000e+00, 1.10935803e+03, 5.39511837e+02, 0.00000000e+00], [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00], [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]]), 'P3': array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 1.]]), 'R0_rect': array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]), 'Tr_velo_to_cam': array([[ 0.01420342, -0.99988233, 0.00579532, -0.0206852 ], [-0.00613689, -0.00588297, -0.99996386, -0.13853743], [ 0.99988029, 0.01416735, -0.00621973, -0.11220291], [ 0. , 0. , 0. , 1. ]]), 'Tr_imu_to_velo': array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 1.]])}, 'annos': {'name': array(['car', 'bicycle', 'pedestrian', 'car', 'car', 'car', 'car', 'car', 'car', 'car', 'pedestrian', 'car', 'car', 'car', 'car', 'car', 'car'], dtype='<U10'), 'truncated': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), 'occluded': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'alpha': array([-10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10.]), 'bbox': array([[-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [ 1.64273e+03, 5.01890e+02, 1.92000e+03, 6.66310e+02], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [ 8.47790e+02, 5.89440e+02, 9.12910e+02, 6.42360e+02], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [ 1.48642e+03, 5.85350e+02, 1.74070e+03, 6.55730e+02], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [ 1.60160e+03, 5.77040e+02, 1.88507e+03, 6.57740e+02], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [ 9.82220e+02, 6.00900e+02, 1.00981e+03, 6.23580e+02], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00], [-1.00000e+00, -1.00000e+00, -1.00000e+00, -1.00000e+00]]), 'dimensions': array([[4.43, 1.7 , 2.09], [1.81, 2. , 0.94], [0.89, 2. , 0.94], [6.12, 2.7 , 2.38], [4.68, 1.8 , 2. ], [4.61, 1.9 , 1.98], [4.86, 1.6 , 1.84], [4.88, 1.6 , 2.07], [5.05, 1.9 , 1.87], [4.5 , 1.6 , 1.98], [0.89, 2. , 0.82], [4.81, 2. , 2.01], [5.1 , 1.9 , 2.32], [4.79, 1.6 , 2.23], [4.01, 1.5 , 1.85], [6.86, 2.5 , 2.04], [4.62, 1.8 , 1.97]]), 'location': array([[-6.859e+01, 2.300e+00, 2.629e+01], [ 3.570e+00, 1.710e+00, 1.490e+00], [ 1.318e+01, -1.030e+00, -4.784e+01], [ 1.597e+01, 2.140e+00, 1.967e+01], [ 3.449e+01, 2.590e+00, 2.586e+01], [-2.920e+00, 3.750e+00, 4.163e+01], [-8.000e-02, 1.300e+00, -1.158e+01], [ 1.576e+01, 2.750e+00, 2.697e+01], [ 7.810e+00, 1.820e+00, 1.630e+00], [ 1.648e+01, 2.440e+00, 2.354e+01], [ 1.279e+01, -7.300e-01, -4.687e+01], [ 2.808e+01, 2.150e+00, 1.934e+01], [ 3.130e+00, 7.250e+00, 9.665e+01], [-8.445e+01, 1.610e+00, 1.854e+01], [ 4.130e+01, 2.280e+00, 2.241e+01], [ 4.630e+01, 1.910e+00, 1.829e+01], [-2.970e+01, 1.980e+00, 1.289e+01]]), 'rotation_y': array([-3.09, -1.54, -0.66, -3.11, -3.06, 1.6 , -1.57, -3.11, -1.21, -3.1 , -0.5 , -3.08, -1.54, 0.03, -3.1 , -3.1 , 0.02]), 'score': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), 'index': array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], dtype=int32), 'group_ids': array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], dtype=int32), 'difficulty': array([-1, -1, -1, 0, -1, 0, -1, 0, -1, 0, -1, -1, -1, -1, -1, -1, -1], dtype=int32), 'num_points_in_gt': array([ 0, 0, 0, 599, 0, 82, 0, 152, 0, 77, 0, 0, 5, 0, 0, 0, 0], dtype=int32)}} Traceback (most recent call last): File "./pytorch/train.py", line 663, in fire.Fire() File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 471, in _Fire target=component.name) File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 681, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "./pytorch/train.py", line 254, in train multi_gpu=multi_gpu) File "/content/second.pytorch/second/pytorch/builder/input_reader_builder.py", line 76, in build multi_gpu=multi_gpu) File "/content/second.pytorch/second/builder/dataset_builder.py", line 64, in build db_sampler = dbsampler_builder.build(db_sampler_cfg) File "/content/second.pytorch/second/builder/dbsampler_builder.py", line 31, in build sampler = DataBaseSamplerV2(db_infos, groups, db_prepor, rate, grot_range) File "/content/second.pytorch/second/core/sample_ops.py", line 25, in init db_infos = db_prepor(db_infos) File "/content/second.pytorch/second/core/preprocess.py", line 106, in call db_infos = prepor(db_infos) File "/content/second.pytorch/second/core/preprocess.py", line 60, in call return self._preprocess(db_infos) File "/content/second.pytorch/second/core/preprocess.py", line 92, in _preprocess for info in db_infos[name]: TypeError: list indices must be integers or slices, not str

[chrisakatibs]

So after passing in a dictionary that it liked for the first error, the above was the next error. Showing that unless I get the structure right or am missing something in generation, the errors will cascade throughout processing.