No corners could be extracted for high resolution images

[Jiloc]

Hello,

I am trying to calibrate a RealSense camera with a MindVision MV-SUA500M-T. The MV camera has a resolution of 2592x1944. I tried AprilGrid targets of all sizes. Now I am using small 3x3 grids with 1cm tags and 0.3cm spacing.

Anyway the problem is that for every dataset I recorded with the MindVision I always receive the error

[FATAL] [1572255263.563560]: No corners could be extracted for camera /cam2/image_raw! Check the calibration target configuration and dataset. 

Tracing back the problem, seems like the problem is the function bool GridDetector::findTargetNoTransformation(const cv::Mat & image, const aslam::Time & stamp, GridCalibrationTargetObservation & outObservation) const which always returns a negative result.

The strange fact is that if I resize the same images by half their size, for most images kalibr is able to find the targets. Like the image below.

So I am wondering, is there a maximum resolution limit or any known problem with small targets?

[Jiloc]

Another question about the Aprilgrid.

The RealSense camera i am using has a wide field of view while the MindVision has a really narrow one. So I supposed that the board to use should be big enough to be clearly recognized by the former camera, but small enough to enter inside the FoV of the latter.

AprilTag can be partially detected, but is it necessary for the whole board to appear in the field of view in at least a few frames? Or could I use boards bigger than the FoV of the MindVision camera and expect kalibr to obtain the intrinsics and extrinsics just using the visible tags?

[jonra1993]

Hi @Jiloc did you able to run realsense cameras with kalibr. I created a ROS bag file with 3 cameras 435 model in a bag file, but when I run kalibr I get similar issue like you.

[Jiloc]

Currently I am using an old SR300, anyway the first times I received your same error also with it. But it was because I was using targets too small for his infrared camera.

Using a bigger target, sometimes, the realsense dataset returns

[ERROR] [1572267009.562248]: initialization of focal length for cam with topic /cam0/image_raw failed  
Projection initialized to: [ nan  nan  nan  nan]
Distortion initialized to: [ nan  nan  nan  nan]

but for most of acquisitions it is able to elaborate the intrinsics.

In my case I have bigger problems with the other camera, and the big question is why kalibr is able to detect the targets if i resize the images...

Anyway I think it would be really helpful if kalibr could accept known intrinsics (like in the realsense case) to elaborate just extrinsics.

[jonra1993]

Thanks @Jiloc I will try with a bigger target. Do you know if there is any problem if image topics are RGB or they should be in gray scale?. Do you know which is the correct camera model for Intel cameras? I agree with you Intel realsense default values work great. There should be an option just for calibrating extrinsic values.

[Jiloc]

I am not sure about but both things, but in this (old) blog post it was written that RGB images could not work and grayscale were needed. About the distortion, I think it should be pinhole-radtan models.

[Jiloc]

Update: resizing the images by 1/4 (2592x1944 -> 648x486) I am able obtain both intrinsics and extrinsics.

Here is the results:

Calibration results 
====================
Camera-system parameters:
	cam0 (/cam0/image_raw):
	 type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>
	 distortion: [-0.0688863  -0.34089068 -0.00581786  0.01892162] +- [ 0.04580006  0.37799349  0.0026342   0.00349984]
	 projection: [ 474.31793194  477.51849871  346.87279913  244.36439318] +- [ 2.15194643  2.15126882  5.93710635  4.11632672]
	 reprojection error: [0.000103, 0.000142] +- [0.429854, 0.581714]

	cam1 (/cam3/image_raw):
	 type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>
	 distortion: [  0.43031325 -29.30408329   0.00247327   0.00085308] +- [ 0.06700969  0.02947162  0.00141929  0.00135467]
	 projection: [ 3808.48988167  3826.46289524   304.44300251   233.32685112] +- [ 0.6173692   0.62248711  0.02577137  0.04606963]
	 reprojection error: [-0.000067, -0.000040] +- [0.427588, 0.493526]

	baseline T_1_0:
	 q: [-0.03972277 -0.23215463  0.00057028  0.97186728] +- [ 0.00763854  0.01227459  0.00411726]
	 t: [-0.07072867  0.00763452  0.20194455] +- [ 0.00030395  0.00007482  0.00065247]



Target configuration
====================

  Type: aprilgrid
  Tags: 
    Rows: 3
    Cols: 3
    Size: 0.01 [m]
    Spacing 0.003 [m]

Still, I suppose I can't use the results with for the original stream. But I would be happy if someone could prove me wrong!

In case someone will be able to check it, I uploaded the bag and the target configuration files used. The bag file contains 4 topis:

/cam0/raw_image -> RealSense SR300 infrared stream (640x480) /cam1/raw_image -> MindVision stream (2592x1944) /cam2/raw_image -> MindVision stream resized by 1/2 (1296x972) /cam3/raw_image -> MindVision stream resized by 1/4 (648x486)

to obtain a successful calibration:

kalibr_calibrate_cameras --target april_3x3_1cm.yaml --bag april_3x3_1cm.bag --models pinhole-radtan pinhole-radtan --topics /cam0/image_raw /cam3/image_raw

while trying to use the images resized by 1/2:

kalibr_calibrate_cameras --target april_3x3_1cm.yaml --bag april_3x3_1cm.bag --models pinhole-radtan --topics /cam2/image_raw

Initializing cam0:
	Camera model:	  pinhole-radtan
	Dataset:          april_3x3_1cm.bag
	Topic:            /cam2/image_raw
	Number of images: 435
Extracting calibration target corners
  Progress 435 / 435 	 Time remaining: 0s                      
  Extracted corners for 234 images (of 435 images)

...
...

Calibration complete.

[ WARN] [1572409121.430502]: Removed 0 outlier corners.

Processed 234 images with 1 images used
Camera-system parameters:
	cam0 (/cam2/image_raw):
	 type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>
	 distortion: [ nan  nan  nan  nan] +- [ nan  nan  nan  nan]
	 projection: [ nan  nan  nan  nan] +- [ nan  nan  nan  nan]
	 reprojection error: [nan, nan] +- [nan, nan]

It is even worst for the original /cam1/image_raw where it gives the error reported in the first message

kalibr_calibrate_cameras --target april_3x3_1cm.yaml --bag april_3x3_1cm.bag --models pinhole-radtan --topics /cam1/image_raw

Initializing cam0:
	Camera model:	  pinhole-radtan
	Dataset:          april_3x3_1cm.bag
	Topic:            /cam1/image_raw
	Number of images: 435
Extracting calibration target corners
  Progress 435 / 435 	 Time remaining:                          
[FATAL] [1572409433.909929]: No corners could be extracted for camera /cam1/image_raw! Check the calibration target configuration and dataset.

[MohrMa2]

Hi I am calibrating 7 FullHD cameras. So no problem with resolution there. You can see with --show-extraction, which corners are extracted.

BTW, in your target the letters are overlapping the corners, which might be a problem for the april tag detector.

[kannidekan]

execuse me, do you know how could i get the IMU YAML file of the requirements in the camera-imu calibration? #341

Thanks very much!

[oafolabi]

I've also encountered this problem. It should work if you move backwards. I think for some reason it is unable to detect the tags if the board is too close to the camera (but this is just my guess)

[KBhanup]

Hi @Jiloc did you able to run realsense cameras with kalibr. I created a ROS bag file with 3 cameras 435 model in a bag file, but when I run kalibr I get similar issue like you.

Hi @jonra1993 ; from time you have posted, I believe you might have found solution; but I am leaving a reply here as I did face the same issue and couln't figure out exact reason for long time; but now its finall resolved and working; I faced 'No corners could be extraced from camera' issue too and when I converted camera recording from RBG to Gray scale recording it worked. Make sure also to change the rosbag recording command to new Grayscale topic.

[aztennenbaum]

hi is there any update on this? it still seems to be failing on the original images whereas downscaled images seem to work

[zoldaten]

1. the main problem here is that Aprilgrid is up-side-down. I have the same problem with custom-generated Aprilgrid with 8x6 board. And couldnt fix that until i download 6x6 grid from kalibr site, printed it on A4 paper. Check axis X,Y orientation and shape of Aprilgrid!
2. extract bag with kalibr_bagextractor --image-topics /usb_cam/image_raw --output-folder . --bag usb_static.bag *put your topic and bag name and see what went wrong.

[heijieke]

hi is there any update on this? it still seems to be failing on the original images whereas downscaled images seem to work

i downscaled images to 640*480,but still failing.what size do you downscaled?

[zoldaten]

320x240. have a look at result - https://youtu.be/9KBqRaBjKgA

i recorded rosbag very-very slow moving about 5 min long. use --show-extraction when invoke kalibrate script to see where kalibr missed images

[ajaykumaar]

Hi @KBhanup, I'm still facing the same issue. My images are all in grayscale (480x480) and I still get the same error. Any ideas how to fix this or would resizing to a smaller dimension help?

Thanks in advance!

[Alex-Beh]

I am facing the similar issue. The reason is I think is the algorithm expecting a AprilTag calibration board which start with marker ID 0. So when I am using the aprilTag board from marker ID 40 to marker ID 79, no corners could be extracted.

[KBhanup]

Hi @KBhanup, I'm still facing the same issue. My images are all in grayscale (480x480) and I still get the same error. Any ideas how to fix this or would resizing to a smaller dimension help?

Thanks in advance!

Hi @ajaykumaar later I also found that, there should be time sync between imu and camera, data collection as the Kalibr software expects them to be in sync. But this was not possible with the rolling shutter (web cam) that I used.