Errors in pointcloud-to-image reprojection

[jaime-spencer-oxb]

Hi, thanks for creating this dataset, it seems very interesting! 😄

I've been looking at projecting the lidar points onto the images to generate depth maps, but have been seeing some errors while following the tutorial.

When projecting points in frames where the vehicle is static the quality of the reprojection is really high, which shows that the lidar-image calibration is very accurate.

However, when the car is in motion, I have noticed that the reprojections are incorrect (particularly on the left image border). Below I've attached a couple of examples. This seems happen even when the vehicle is travelling forward in an open street (i.e. no urban canyon).

Do you think this is due to incorrect pose estimates? Or could it be related to the pointcloud motion compensation?

As you mention in the paper, using the relative pose instead of the global pose would likely be more accurate for short timescales. Are those poses available in the dataset? As far as I can tell, there is only the raw IMU data.

Thank you!