lidar_camera_calibration icon indicating copy to clipboard operation
lidar_camera_calibration copied to clipboard

Published 20 hours ago • verified publisher icon ankitdhall

monocular camera image with Lidar Points

Open SinghiDivyanshu opened this issue 5 years ago • 11 comments

hi,

what is the next step after finding the rigid body transformation using Velodyne LIDAR and monocular camera. my RMSE value is 0.1636 as shown in the image.

so how to find the image with LIDAR point cloud on it. output_camera_calibration

SinghiDivyanshu avatar Jan 13 '20 12:01 SinghiDivyanshu

Hi @SinghiDivyanshu, once you have the rigid body 4x4 transformation matrix you'll have to perform the following steps:

  1. Transform the point cloud captured from LiDAR to Camera Co-ordinate system by using the 4x4 Rigid Body Transformation matrix obtained from this package.
  2. Project the Points from Camera Co-ordinate system to image plane by using 3x4 Camera Instrinsic / Projection Matrix which contains camera info such as focal lengths etc.

Let me know if you have doubts.

SiddhantNadkarni avatar Apr 26 '20 09:04 SiddhantNadkarni

Hi @SiddhantNadkarni ,

How you do transform the point cloud to the camera coordinate system and then project it? I don't know what packages or code to use.

AnthonyLadewig avatar Nov 02 '20 10:11 AnthonyLadewig

Can you please mention what changes you had to make in the launch and lidar_camera_calibration.yaml to get it running, my topic names are different and I'm not able to understand which change has to be made where.

dkhanna511 avatar Nov 05 '20 05:11 dkhanna511

@dkhanna511 In the launch file, you have to uncomment the code under aruco mapping and then change the remap topic.

remap from="image_raw" to="zed/left/image"

Change the zed/left/image to your camera's topic.

In the yaml file, do the same.

To find your camera's topics, run the launch for your camera and do 'rostopic list' in the terminal.

AnthonyLadewig avatar Nov 05 '20 05:11 AnthonyLadewig

@AnthonyLadewig
Hi, thanks for the response. I did these changes, Now I'm facing an openCV issue. that is mentioned in #131. Do you have idea about how to solve that? Have you encountered this issue?

dkhanna511 avatar Nov 05 '20 09:11 dkhanna511

@dkhanna511 Unfortunately I have not had this problem and do not know how to fix it. Sorry

AnthonyLadewig avatar Nov 05 '20 11:11 AnthonyLadewig

@dkhanna511 Unfortunately I have not had this problem and do not know how to fix it. Sorry

@AnthonyLadewig I ran the same process on another laptpo and it works. Apparently there are some OpenCV conflict issues. After running the find_transform.launch file, only mono8 window is opening. I have not created the setup as of now but I want to know if the polygon window and point cloud window pops up when there is no marker at first.

dkhanna511 avatar Nov 06 '20 08:11 dkhanna511

@dkhanna511 from my experience, the polygon/point cloud window doesn't come up if the lidar isn't running.

AnthonyLadewig avatar Nov 06 '20 15:11 AnthonyLadewig

@dkhanna511 from my experience, the polygon/point cloud window doesn't come up if the lidar isn't running.

@AnthonyLadewig The lidar is running. I can see the point cloud fees on RVIZ

dkhanna511 avatar Nov 09 '20 08:11 dkhanna511

@dkhanna511 Im not sure what the problem might be then. Is the calibration software actually subscribing to the lidar topic? What lidar are you using btw?

AnthonyLadewig avatar Nov 10 '20 13:11 AnthonyLadewig

@AnthonyLadewig I was using VLP16. I had to install everything from scratch and now all the 3 windows come up.

dkhanna511 avatar Nov 11 '20 09:11 dkhanna511