Improve results further

[poornimajd]

Hello I tried lidar camera calibration using a checkerboard in different poses,marking 5 points in each pose,for both camera and point_cloud.But I get the following reprojection- I also tried with a cardboard with a cutout,so as to get more points in each pose. But the results did not improve much,as shown below: I have limited the fov of lidar while calibrating,does that have any effect on the results? I was unable to Refine pose estimates using LM because of some error ,which may be due opencv version problem. Is there any other way to improve the results? Any suggestion is appreciated. Thanks

[heethesh]

• What was the total number of correspondences you used for calibration?
• Place the calibration target in multiple positions such that the target covers all regions and extremes of the image (see the demo video).
• Ensure that you pick points on the rectified image (the image displayed in the GUI). The launch file rectifies the image using the camera info topic, so make sure that the image is not rectified twice.
• Most likely the version of OpenCV that your ROS version uses does not have the LM refinement method, you can try to separately refine the pose in Python later (using the saved correspondences and initial results) without using ROS OpenCV. To install OpenCV, pip install opencv-python and remove the source /opt/ros/distro command from your .bashrc, and try it on a new terminal - you should be able to work with a newer version of OpenCV.

[poornimajd]

Thanks for the detailed reply!!

* What was the total number of correspondences you used for calibration?

I tried with 5-8 points with different poses such that it covers extremes of the camera image.

• Ensure that you pick points on the rectified image (the image displayed in the GUI). The launch file rectifies the image using the camera info topic, so make sure that the image is not rectified twice.

• Most likely the version of OpenCV that your ROS version uses does not have the LM refinement method, you can try to separately refine the pose in Python later (using the saved correspondences and initial results) without using ROS OpenCV

I will check this out.Thanks for the suggestion!

[Bsting118]

How would you check to make sure the image is not rectified twice? Also, how would you separately refine the pose in Python since I am having almost the same issue with OpenCV and having an older version that does not have the solvePnPRefineLM method.

[heethesh]

@Bsting118

How would you check to make sure the image is not rectified twice?

After you have performed your camera calibration, ROS will auto save this data in ~/.ros/your_camera.yaml and when you playback the camera image topic (eg: /your_camera/image_raw) using rosbag play, ROS will publish this saved calibration on the camera info topic corresponding to the same camera name (eg: /your_camera/camera_info). Take a look at display_camera_calibration.launch and make sure the rectified image here looks exactly same to the image displayed in the GUI to pick 2D points.

Also, how would you separately refine the pose in Python since I am having almost the same issue with OpenCV and having an older version that does not have the solvePnPRefineLM method.

You can simply create a Python virtual env and pip install opencv-python there (no need to worry about the OpenCV C++ version installed on your system). Just load the correspondences which are saved in the calibration folder and use the estimated extrinsics to run the refinement. FYI, the current master checks for the presence of this method and skips the refinement, if not.

[Bsting118]

@heethesh When making a python virtual env in my workspace directory/folder, should I be using Python3 or Python? My CV2 seems to be in my Python so would I use Python? Or use Python 3(for python virtual env)?

[yulan0215]

Thanks for the detailed reply!!

* What was the total number of correspondences you used for calibration?

I tried with 5-8 points with different poses such that it covers extremes of the camera image.

• Ensure that you pick points on the rectified image (the image displayed in the GUI). The launch file rectifies the image using the camera info topic, so make sure that the image is not rectified twice.

• Most likely the version of OpenCV that your ROS version uses does not have the LM refinement method, you can try to separately refine the pose in Python later (using the saved correspondences and initial results) without using ROS OpenCV

I will check this out.Thanks for the suggestion!

Did u solve or figure out the problem? I have same problem as you...

[poornimajd]

I took a larger checkerboard and while reprojecting the lidar points make sure that you have changed the parameters of the extrinsic matrix as obtained by your method.

[HRex39]

Thanks for the detailed reply!!

* What was the total number of correspondences you used for calibration?

I tried with 5-8 points with different poses such that it covers extremes of the camera image.

• Ensure that you pick points on the rectified image (the image displayed in the GUI). The launch file rectifies the image using the camera info topic, so make sure that the image is not rectified twice.
• Most likely the version of OpenCV that your ROS version uses does not have the LM refinement method, you can try to separately refine the pose in Python later (using the saved correspondences and initial results) without using ROS OpenCV

I will check this out.Thanks for the suggestion!

Hello, I have tried multiple poses(10 + ) and picked points using rectified image. But each time I tried, the program return difference result in rotation but approximate value in translation.
I want to know, the more points the program uses, shouldn't the rotation matrix be closer to a fixed value？Why it the rotation matrix become vary?