TrackID inconsistent across across cameras and lidar

[srikanthmalla]

Hi, I found out that the track-ids for same object across different sensors (lidar and camera) at same timestep. Across different timesteps, across different cameras is not same as shown in this video: https://www.youtube.com/watch?v=YjOUamkRBRo

Is there a way I can get the same track ID for those both cases. Please let me know.

Best Regards, Srikanth

[henrikkretzschmar]

Hi Srikanth,

The objects were labeled independently in lidar and in the various cameras. The dataset does not provide ground truth correspondences between lidar and camera nor between the various cameras. You could, however, try to estimate the correspondences by associating the camera labels with the projected lidar labels for each sequence. Hope this helps.

Best, Henrik

[srikanthmalla]

Hi Henrik, Thanks for the prompt reply. But it will be prone to errors because of the following issues:

1. 3D bounding box when projected on image and when converted to 2D box using min max of projected vertices won't be tight.
2. Since the 2D boxes are annotated considering occlusion (dimensions won't be fixed in all cases), but not for the 3D boxes.
3. Also when there are objects are very close-by, they can have more IOU with this big projected 3D box (can cause mismatch).

So, I think there will be issues with close-by objects or crowded pedestrians associating boxes.

Option 1: I can think of directly using projected 3D box (after converting it to 2D box and using it). But the problem with it will be it won't consider occlusion, also I think it won't be tight like the annotated 2D boxes. https://www.youtube.com/watch?v=ekizQEwPqew

Option 2: May be we need to use temporal info as well for associating, not just single time step to avoid those mismatches.

Let me know if what I mentioned is correct. If it is correct, do you guys have any feedback or suggestions to use above or any other options.

Best, Srikanth

[henrikkretzschmar]

Hi Srikanth,

You will probably want to use temporal information indeed. We experimented with a simple IoU-based association and obtained reasonable results. We computed the best associations based on the entire segments. Note, though, that we didn't conduct a thorough analysis of this approach. See #141 for a related discussion.

Best, Henrik

[srikanthmalla]

Hi Henrik, That's great to hear. Is it possible for you to post that mapping here (in some text file or something): For example: xxx1_track_id(cam 1) : yyy1_track_id_lidar xxx2_track_id(cam 2) : yyy2_track_id_lidar

I am pretty sure this will help everyone as people don't have to repeat the same process of associating again and again (if they are planning to use multiple sensor information). Please let me know.

Best, Srikanth

[henrikkretzschmar]

Hi Srikanth,

I'm afraid the association is currently not part of the dataset, and we won't be able to provide it here, unfortunately. We may be able to include it in a future version of the dataset.

I hope that the discussion in #141 will give you some interesting ideas of possible directions to explore.

Best, Henrik

[haibao-yu]

Hi, I found out that the track-ids for same object across different sensors (lidar and camera) at same timestep. Across different timesteps, across different cameras is not same as shown in this video: https://www.youtube.com/watch?v=YjOUamkRBRo

Is there a way I can get the same track ID for those both cases. Please let me know.

Best Regards, Srikanth

Hi, I'm creating the tracking dataset and I'm learning how to deal with the tracking ids interrupted by occlusion from waymo dataset. So can you point out what time the case that track-ids for same object is not same across different timestamps occurs in this video (it is too long) ? Thanks!