nlminb in estimateUndistortion

[JanJaapPoos]

While trying to run calibrateCameras() on the example videos, there is a call to nlminb() in the estimateUndistortion() function that seems to halt the program. Specifically, this happens for p == 3

so that p_start[[3]] equals c(0.1, 0.01, NA, NA, NA),

when calling

nlminb(start=c(image.size[1]/2, image.size[2]/2, p_start[[i]]), objective=undistortionError, coor.img=coor.2d, coor.obj=coor_obj_array, image.size=image.size)

I see that in the undistortionError() function the NAs get translated to 0, and if I set p_start[[3]] to c(0.1, 0.01, 0, 0, 0), then nlminb() indeed tries to minimize the objective. However, then the interpretation of what the minimizer does also changes. I am not sure what makes the nlminb() hang, because even setting control = list(trace = 1) gives only iteration 0. This is under R4.4.1 on a windows computer.

The output until the point that the calibrateCameras() stops working is:

calibrateCameras

Checking input parameters...
	Calibration file: 'calibration.txt'
	Calibration input type: video
	Number of camera views: 2
	One view upside-down?: FALSE
	Calibration videos:
		Filenames:
			Left.mov   (200 frames, 1920x1080)
			Right.mov  (200 frames, 1920x1080)
		Square size: 63.42 mm
		Internal corners: 8 x 4 (32 total)

Calibration checkerboard corner detection...Saved calibration corners found for all views in 'Corners' folder.

	Left.mov  : Corners detected in 81 frames
	Right.mov : Corners detected in 81 frames

	Do you wish to re-detect the calibration corners? (y/n) : n

	Number of cases in which corners were found in at least 2 views: 69

Estimating undistortion parameters...

I am hoping that you can look into this. I am really impressed with how smoothly the package worked otherwise. Many thanks in advace

[aaronolsen]

Thank you for posting this error @JanJaapPoos and your kind works on the package! Unfortunately, I don't have any grant funding and am no longer in an academic position, so in terms of error troubleshooting I can only offer quick suggestions, not more helpful but time-consuming debugging (a con of free and open source...). But it's great that the issue is posted here. If you or another person can come up with a fix, please feel free to fork and I can update the package!

[JanJaapPoos]

Hi @aaronolsen. Thanks for the swift reply. I forked the code and now understand how to make a fix. In short there are three fixes possible: 1) never estimate the two p parameters in undistort() and set them to zero. In practice, that what the code was doing. 2) always estimate the two p values in undistort, starting at zero, 3) mimic the old code in which p values could be estimated, or fixed, with the current code fixing them,

The first option means that the call to undistort() in undistortionError and distortionError fixes the two p parameters to zero:

undistort(coor.img, image.size=image.size, center=c(p[1], p[2]), k=c(p[3], p[4], p[5]), p=c(0, 0))

This means that these two parameters are always fixed at zero and never estimated, and my impression of the code is that this was the case anyway (because the last two elements of all vectors with starting values in estimateDistortion() were always NA. I will write out option 3, and commit it to my fork. Having been able to continue my exploration of the package, I continue to be impressed. I have a few questions about the outputs, but I'll probably find out, or post a separate issue. I intend to use the package for educational purposes.