How to get point clouds from all lidars for a frame in v2 devkit

[SM1991CODES]

Hi, I followed the v2 tutorial and was able to get point clouds for individual lidar sensors as follows:

`def generate_trainval_dataset(path_dataset_root, frame_step, max_frames_save): """ Function generates BEV frames from base Waymo dataset """

components = {"lidar": "lidar", "lidar_calibs": "lidar_calibration", "lidar_labels": "lidar_box", "lidar_pose": "lidar_pose", "lidar_seg": "lidar_segmentation", "ego_pose": "vehicle_pose", }

path_pcl = path_dataset_root + "/" + components["lidar"] path_labels = path_dataset_root + "/" + components["lidar_labels"] path_sem = path_dataset_root + "/" + components["lidar_seg"] path_calibs = path_dataset_root + "/" + components["lidar_labels"] path_ego_pose = path_dataset_root + "/" + components["ego_pose"]

frames_pcl = os.listdir(path_pcl) n_frames_pcl = len(frames_pcl) - 1 # there's a metadata file, don't know yet if this can be deleted

frames_labels = os.listdir(path_labels) n_frames_labels = len(frames_labels)

frames_sem = os.listdir(path_sem) n_frames_sem = len(frames_sem)

for index in range(0, n_frames_pcl, frame_step):

context_name = frames_pcl[index]
pcl_path = path_pcl + "/" + frames_pcl[index]
labels_path = path_labels + "/" + frames_labels[index]

lidar_df = read(dataset_train_dir_root, components["lidar"], context_name)
lidar_labels_df = read(dataset_train_dir_root, components["lidar_labels"], context_name)
lidar_calibs_df = read(dataset_train_dir_root, components["lidar_calibs"], context_name)

df_laser_calibs = v2.merge(lidar_df, lidar_calibs_df)
df = v2.merge(df_laser_calibs, lidar_labels_df)

for index, data_row in iter(df.iterrows()):
#   print(data_row)

  lidar = v2.LiDARComponent.from_dict(data_row)
  print(lidar.key.laser_name)
  lidar_calib = v2.LiDARCalibrationComponent.from_dict(data_row)

  pcl = v2.convert_range_image_to_point_cloud(range_image=lidar.range_image_return1, calibration=lidar_calib)
  #viz_utils.draw_lidar(pcl)`

However, the above only returns all TOP lidars at first, then FRONT lidar and so on.

I wish to create a single point cloud out of all 5 lidars for each frame (and collect objects for the frame).

What is the most efficient way to do that?

I have alsso tried the tfrecords format and I can do this there easily. However, tfrecord loading is extremely slow and impractical.

Please help

[JingweiJ]

Hi @SM1991CODES ,

For each run segment / context, you may execute dask.dataframe.DataFrame.compute to actually read the data of your dataframe first (or a sub-dataframe by key matching if you just want a part of the context, e.g. one timestamp). The key.laser_name and key.frame_timestamp_micro columns can help you select the rows you want to concatenate together.

Be careful that the laser names of your lidar and lidar calibration must match before calling convert_range_image_to_point_cloud. See https://github.com/waymo-research/waymo-open-dataset/issues/656 .