Question about the 3D annotation for fully occlusion case

[haibao-yu]

Hi, Waymo dataset:

Thanks for your great dataset. I know you have labelled the objects with 3D information and tracking ID which are fully occluded by other objects. I wonder to know in which sensor you consider the occlusion label, Camera or LiDAR? Further, how do you label the fully occluded objects?

Thanks, Haibao

[VincentCa]

Hi Haibao,

In Waymo Open Dataset, the 5 cameras are closely located to the top (main) LiDAR. Therefore, you can assume that the occlusion patterns between these to be quite similar. The LiDARs located on the sides of the vehicle do have a different viewpoint, and some of the labels may be based solely on their returns. If you want to know more about the sensor location, you can look at the extrinsic calibration. Therefore:

• If you want to determine whether a labeled object is likely occluded from the collective LiDAR's point of view, we recommend using num_lidar_points_in_box (which factors in point counts from all LiDAR's).
• If you want to determine whether a labeled object is likely occluded from the camera point of view, we recommend using num_top_lidar_points_in_box (which only factors in point counts from the top/main LiDAR).

Without going into specifics about the occlusion labeling process, remember that we have accurate ego poses which enables a variety of interpolation techniques.

Hope this helps!

[haibao-yu]

Hi, VincentCa:

Thanks for your nice reply. You have solved my question. There is another related technique question about the trajectory definition: why should we label the fully occluded objects? I mean that we can regard the trajectory involving the occlusion as two separate trajectories and discard the occluded parts.

Thanks, Haibao

[VincentCa]

Hi Haibao,

Glad this was helpful! Regarding your follow-up question, we do believe that re-identification is valuable in practice as it allows for better reasoning about the history of a given object. Of course this can be technically challenging, but an ideal method would be able to merge the tracks/trajectories.

Best, Vincent

[haibao-yu]

Hi, Vincent: Thanks your helpful reply!