DLT algorithm needs at least 6 points for pose estimation from 3D-2D point correspondences

[xuhao1]

Hi there, I am trying to calibrate a four-camera system. I use an AprilTag grid for calibration. However, when I run the command it gives

root@17ae5990c981:/catkin_ws# rosrun kalibr kalibr_calibrate_cameras     --bag /data/quad_cam_calib-split.bag --target /data/aprilgrid.yaml     --models omni-radtan omni-radtan omni-radtan omni-radtan  --approx-sync 0     --topics /arducam/image_0/compressed /arducam/image_1/compressed /arducam/image_2/compressed /arducam/image_3/compressed
importing libraries
Initializing cam0:
	Camera model:	  omni-radtan
	Dataset:          /data/quad_cam_calib-split.bag
	Topic:            /arducam/image_0/compressed
	Number of images: 1916
Extracting calibration target corners
  Extracted corners for 683 images (of 1916 images)                              
Traceback (most recent call last):
  File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_cameras", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_cameras", line 450, in <module>
    main()
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_cameras", line 187, in main
    if not cam.initGeometryFromObservations(observations):
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py", line 57, in initGeometryFromObservations
    success = self.geometry.initializeIntrinsics(observations)
RuntimeError: OpenCV(4.2.0) ../modules/calib3d/src/calibration.cpp:1171: error: (-2:Unspecified error) in function 'void cvFindExtrinsicCameraParams2(const CvMat*, const CvMat*, const CvMat*, const CvMat*, CvMat*, CvMat*, int)'
> DLT algorithm needs at least 6 points for pose estimation from 3D-2D point correspondences. (expected: 'count >= 6'), where
>     'count' is 5
> must be greater than or equal to
>     '6' is 6

Why is this case? Thanks!

==========================

I found https://github.com/ethz-asl/kalibr/blob/master/aslam_cv/aslam_cameras/include/aslam/cameras/implementation/OmniProjection.hpp#L922

should be

if (Ps.size() < 6) {

[goldbattle]

Try visualizing the extraction. Sounds like there are not enough points being tracked / extracted in the bag. Try testing with a single cam, then move to more cameras.

[xuhao1]

Hi @goldbattle I open a pull request https://github.com/ethz-asl/kalibr/pull/556 for fixing this issue.

[goldbattle]

The function should work for 4 points since the target is planar: https://github.com/ethz-asl/kalibr/blob/e9453e0734abec221d4335b1d6fb3202b1c9ff2b/aslam_cv/aslam_cameras/src/GridCalibrationTargetObservation.cpp#L38-L61

This check seems to be failing which is causing it to try to treat the 3D points as not being planar. https://github.com/opencv/opencv/blob/8b4fa2605e1155bbef0d906bb1f272ec06d7796e/modules/calib3d/src/calibration.cpp#L1108-L1115

It is a bit hard to read the OpenCV code, but this might be due to the SVD condition... Could you provide the bag where this fails for you? Or what does the frame look like which it fails on?

[xuhao1]

Hi, @goldbattle Thanks for your reply. Here is my bag and I use this command

source devel/setup.bash
rosrun kalibr kalibr_calibrate_cameras \
    --bag /data/quad_cam_calib-split.bag --target /data/aprilgrid.yaml \
    --models omni-radtan omni-radtan omni-radtan omni-radtan  --approx-sync 0 \
    --topics /arducam/image_0/compressed /arducam/image_1/compressed /arducam/image_2/compressed /arducam/image_3/compressed

The calibration target is a standard 6x6 April tag. In this bag, I am trying to calibrate a quad-cam-imu system in one bag. The camera is placed 90 deg to each other. So I rotate the cam system and move it again and again. In this case, some images will only capture a few grid points at the corner of the camera plane. I also found in this bag, that the camera intrinsic will be inited to nan. My branch https://github.com/HKUST-Swarm/kalibr/tree/quadcam_imu can calibrate the intrinsic of this bag now, working on extrinsic.

=======

Update: This branch now can calib the intrinsic and extrinsic with

rosrun kalibr kalibr_calibrate_cameras     --bag /data/quad_cam_calib-split.bag --target /data/aprilgrid.yaml     --models omni-radtan omni-radtan omni-radtan omni-radtan --approx-sync 0     --topics /arducam/image_0/compressed /arducam/image_1/compressed /arducam/image_3/compressed /arducam/image_2/compressed

[xuhao1]

Image of the camera system, four 220deg fisheye cameras. In recording the bag, I

1. Making center of each pair of the camera watch the target and moving
2. Making center of each camera watch the target and moving

[goldbattle]

The rosbag is only 2.4s long. Could you double check this?

[xuhao1]

@goldbattle Sorry, I modified the bag mistakely. Now the bag is at full length (with 5749*4 images)

[JzHuai0108]

I also encountered the same issue on tartancalib for omni-radtan. Your pull request may not be the ultimate solution, but it is a quick fix. Note I only changed OmniProjection.hpp which is relevant. As an aside, I also tried to keep only observations with at least 6 points before feeding to the geometry initialization method, but this still triggered that DLT runtime error. Apparently, this needs further investigation.

[TheFloe1995]

I was also facing this issue, and I want to add some information/findings:

• Fisheye camera ~180 deg FoV
• Double Sphere
• Ubuntu 20.04 container
• Tried both apriltag grid and classic checkerboard targets
• Some recordings fail, some work

I just discovered, that it doen't fail with this error when using the Ubuntu 18.04 Docker container. Can anybody confirm?

So maybe it's opencv that somehow changed it's behavior? Ubuntu 18 has OpenCV 3.2 installed, 20 has 4.2 (if I'm not mistaken).

[lintglitch]

@TheFloe1995 thank you for the tip! I have the same issue with the 20.04 docker but it does not happen in the 18.04 docker. It seems to always happen with certain rosbags, while it works just fine with others on 20.04.