How can I train a road-based model using only the road-point cloud of dair-v2x?

[KKHAOO]

trafficstars

How can I train a road-based model using only the road-point cloud of dair-v2x?

[yifanlu0227]

Hi! You can get started with this yaml(opencood/hypes_yaml/dairv2x/Single/DAIR_single_m1.yaml). The input_source: ['camera','lidar'] includes 2 modalities, but it is okay. The camera data will not used actually.

To only use the road-side unit data, you should change 2 parts in existing code.

1. you should change the random selection this line to only selecting RSU agent. The cav_id is 0 for the vehicle and 1 for the RSU.

            if not self.visualize:
                options = []
                for cav_id, cav_content in base_data_dict.items():
                    if cav_content['modality_name'] in self.ego_modality:
                        options.append(cav_id)
                selected_cav_base = base_data_dict[random.choice(options)] # change this line to only select RSU agent.
            else:
                selected_cav_id, selected_cav_base = list(base_data_dict.items())[0]

2. There is an augmentation which permute the cav_id of vehicle and RSU in DAIR-V2X's basedataset code. To make sure cav_id is 0 for the vehicle and 1 for the RSU, you can simply remove those lines.

            if self.train: # randomly choose RSU or Veh to be Ego
                p = np.random.rand()
                if p > 0.5:
                    data[0], data[1] = data[1], data[0]
                    data[0]['ego'] = True
                    data[1]['ego'] = False
            else:
                # evaluate, the agent of ego modality should be ego
                if self.adaptor.mapping_dict[data[0]['modality_name']] not in self.ego_modality and \
                    self.adaptor.mapping_dict[data[1]['modality_name']] in self.ego_modality:
                    data[0], data[1] = data[1], data[0]
                    data[0]['ego'] = True
                    data[1]['ego'] = False

[KKHAOO]

I modified these two parts according to your method. Firstly, I modified the first step you mentioned selected_cav_base = base_data_dict[random.choice(options)] to selected_cav_base = base_data_dict[1], but the visualization result shows the vehicle-side detection results. Therefore, I modified the get_item_test function.I change this line if cav_content['ego']: to if cav_content['ego']==False: ，The final visualization result became the roadside detection results, but there are some issues with the GT boxes. These issues mainly include the following two aspects:

1. There are some gt boxes outside the range of the point cloud that shouldn't be there. I checked the gt boxes after vehicle-road collaboration, and it seems these extra boxes belong to the vehicle-side.
2. Some vehicles are not labeled with GT boxes."

[KKHAOO]

Do you know how to solve these problems?