Timestamps for camera data? Right camera projection?

[pwais]

Thank you for making the velodyne and SICK timestamps available, but do you think you might make the camera timestamps available too? In the original KITTI Tracking & Object Benchmarks, I observed there's about a 10ms difference between velo and camera (which is very nice!). The timestamps would perhaps be a bit more relevant for the side (fisheye) cameras in KITTI-360, especially since it looks like the intention is to use lidar labels for those cameras.

Also, when I draw the lidar points onto the right camera, I notice a bit of shift that looks perhaps due to lag rather than calibration:

Here's the same point cloud plotted on the left camera-- the van in the distance is very tight:

[yiyiliao]

Hi @pwais , we have released the timestamps of all cameras. Please check the latest download scripts for that.

Thanks for pointing out the shift on the right camera. I am not sure what is the cause and will check it out. The time difference between the left and right should be very small (<0.1ms).

[pwais]

Thank you for the timestamps @yiyiliao !

I did some more plotting with the right camera and while I can't get a good alignment, I'm a bit confused about the provided calibration data. Is it best to: (1) use the extrinsics for the right camera (image_01 in calib_cam_to_pose.txt) plus the intrinsics (P_rect_01 in perspective.txt) ? (2) use extrinsic for the left camera to move to left camera frame, then use R_01 and T_01 to in perspective to move to right camera frame, then use P_rect_01 as intrinsics? (3) something else?

I poked around in this repo but I don't think I saw any example using the code for the right camera ...

I think there's some subtle confusion between the projective matrix P and intrinsic matrix K ... in the original KITTI datasets (e.g. Tracking & Object benchmarks), only P was provided (as in P = K[R|T]). In the case of KITTI-360, the code appears to call the projective matrix K yet the docs call it a "perspective intrinsic" so it's a little confusing. Moreover, if the data in perspective.txt for the right camera is indeed P = K[R|T], it appears undocumented as to what the transform [R|T] is (is it from the ego frame? the left camera? something else? is it to the rectified or raw right frame ..?)

It might help to have more precise documentation of perspective.txt.