Camera intrinsics - WRONG calculation

[Witek-]

Issue Summary

The intrinsic parameters are incorrectly calculated. The distortion coefficients values depend very much on the square size, while they should not. The distortions will get smaller as the set square size gets smaller. Also the camera matrix will vary, but the differences are much smaller. The real square size of my calibration pattern is 40 mm.

For 40 mm (absurdly large distortions):

Re-projection error - cv::calibrateCamera vs. calcReprojectionErrors:   0.178398 vs. 0.178398
Intrinsics_K:
[1519.282524723252, 0, 630.7101829172911;
 0, 1518.305732120222, 552.0404151135538;
 0, 0, 1]
Intrinsics_distCoeff:
[-16.21773954167164;
 58.58351920717708;
 0.0004035325834125546;
 0.0007339557064146433;
 60.42666928973438;
 -15.98575103113161;
 54.62808753117094;
 77.30600540156117;
 0;
 0;
 0;
 0;
 0;
 0]

For 0.04 m (most od distortions are reasonable, but some seem large):

Re-projection error - cv::calibrateCamera vs. calcReprojectionErrors:   0.178631 vs. 0.178631
Intrinsics_K:
[1518.917427927563, 0, 626.265203566601;
 0, 1518.055764404161, 554.0151831646898;
 0, 0, 1]
Intrinsics_distCoeff:
[1.46132521242287;
 -1.057816274144218;
 0.0004526096516661285;
 0.0004968042743693283;
 30.86284793196806;
 1.699189826340251;
 -1.129655255489572;
 32.66544431645934;
 0;
 0;
 0;
 0;
 0;
 0]

For a value of 1 (small difference compared to value of 40):

Re-projection error - cv::calibrateCamera vs. calcReprojectionErrors:   0.178399 vs. 0.178399
Intrinsics_K:
[1519.161731011366, 0, 630.7231313632034;
 0, 1518.185048595225, 552.0557603679689;
 0, 0, 1]
Intrinsics_distCoeff:
[-16.21337325210424;
 58.55684620700714;
 0.0004027380651355804;
 0.0007339581398510276;
 60.34030801021383;
 -15.98138505944125;
 54.60259584095242;
 77.2096854475584;
 0;
 0;
 0;
 0;
 0;
 0]

The input images seem to be correct. They all get identified and the orientation of the chessboard is always the same, like shown below.

What is particularly interesting, when I change the --grid_number_inner_corners from "5x4" to "4x5" the results are different! Now the same image looks like that, which I think is correct:

For 40 mm and 4x5 grid (and not 5x4 as in the three previous cases above) the results are definitely not the same as for 40 mm and 5x4 grid:

Re-projection error - cv::calibrateCamera vs. calcReprojectionErrors:   0.178462 vs. 0.178462
Intrinsics_K:
[1519.677522864292, 0, 626.5140403467877;
 0, 1518.813570928729, 553.8010518876664;
 0, 0, 1]
Intrinsics_distCoeff:
[-12.81817673470436;
 36.87139295094519;
 0.0004912281557907267;
 0.0004834319831379921;
 39.61518597810399;
 -12.58819611151161;
 33.73441991392357;
 50.67831072800077;
 0;
 0;
 0;
 0;
 0;
 0] 

It makes no sense to me.

I calibrated the camera with the OpenCV calibration routine, and no matter what square size or grid order I put, I always get these (identical up to 6th or 7th digit):

camera_matrix: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 1.5196168947201249e+03, 0., 6.2746567336559463e+02, 0.,
       1.5188003633364824e+03, 5.5450324138057715e+02, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
   rows: 5
   cols: 1
   dt: d
   data: [ -2.2533666665351371e-01, 1.7334571696344514e-01,
       5.7222134071501255e-04, 4.8970551244651448e-04, 0. ]
avg_reprojection_error: 1.8253178569745693e-01

Can anybody explain what's going on? Am I doing something wrong or there is a bug in the code? Or perhaps the calibration pattern is too small? Even so, the results should not vary. I am puzzled.

Executed Command (if any)

bin\calibration --mode 1 --grid_square_size_mm 40 --grid_number_inner_corners "5x4" --camera_serial_number 1 --calibration_image_dir "E:\robot-kompan\identyfikacja sylwetki ludzkiej\sekwencje\calib_left_3"

Type of Issue

Execution error Help wanted

System Configuration

Operationg system: Windows 10 Graphics card: Nvidia GTX1060 6GB Driver Version: 461.40 from 26.01.2021 CUDA 10.1, CuDNN: 8.0.5 Visual Studio Community 2017 Caffe, OpenCV: Default from OpenPose OpenPose prebuilt 1.6 and 1.7 and master build

[lks90commit]

I'm also having a similar problem on the intrinsic calibration, with strange distortion values (specially for the k parameters). Have you found a solution for your problem?

[Witek-]

I gave up and I reconstruct the 3D keypoint locations on my own in OpenCV and it seems to work

[DanialV]

@Witek- Hi I want to reconstruct the 3D keypoint but unable to use openose 3d module can you explain your approach with OpenCV

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uSerratos]

Does anybody know how to map the camera calibration parameters from openCV to openpose?