dice K1R3-K2R5-K3R7 Distortion Model

Hi, I've been trying to calibrate a microscope for DIC using DICE. Unfortunately, I can't make a good enough checker pattern, so I've generated a radial distortion coefficient using some pure translative measurements and DICe, with the above model, by assuming only K1R3 distortion. Basically using DIce to track the subsets in the distorted images, then undistorting both the original, and final, to regenerate the displacements of the subsets. It worked pretty well, and I wanted to feed this back into the calibration file so I can continue taking data with minimal post-processing, as well as implementing your VSG formulation. However, the calibration in DICe is behaving unexpectedly. I was wondering if you could help.

I made up an .XML file, as below. Then used DICe on some pure translative datasets again, as a test. Unfortunately, the results were somewhat different to my formulation.

The difference between the original coordinates and the 'model' or undistorted coordinates come out the same, which was promising. I feel this indicated we were treating the calibration coefficients the same way. (I treated the image centres slightly different, which is the reason for the 1024px offset)

Here are some images for comparison

But the difference between the distorted and undistorted displacement maps was very different.

Which means the final result turned out a little odd.

I would expect that the displacement correction would look more like mine.

I couldn't see any differences between how we formulated the equations, which shows in the comparisons and similarities between model space and image space; however, I can't quite figure out what steps are in your code to calculate displacement. I'm not very au fait with coding! So I couldn't troubleshoot that aspect.

Your formulation, below, seems to map an undistorted image to a distorted one. And then, as I understand, you iterate over this function until your inversion converges. As I said, we seem to get the same results when moving the reference frame of the reference image (your output of MODEL_COORDINATES), but very different when we do compare displacement.

DICe_Camera.cpp, lines 719-730

    	case K1R3_K2R5_K3R7:
      		for (size_t i = 0; i < vec_size; i++) {
        		x_sen = sen_x[i];
        		y_sen = sen_y[i];
        		rad = sqrt(x_sen * x_sen + y_sen * y_sen);
        		assert(rad!=0.0);
        		dis_coef = k1 * pow(rad, 3) + k2 * pow(rad, 5) + k3 * pow(rad, 7);
        		x_sen = (rad + dis_coef)*x_sen / rad;
        		y_sen = (rad + dis_coef)*y_sen / rad;
        		image_x[i] = x_sen * fx + cx;
        		image_y[i] = y_sen * fy + cy;
     											 }
      break;

Here's what I used, I'm mapping a distorted (image coordinates in x,y) to undistorted (real-world coordinates in x,y) here, so the opposite direction. I use this function on the initial locations, and then on the initial locations + measured displacement {COORDINATES_(X,Y) + DISPLACEMENT_(X,Y)}

K is just k1 from the K1R3 model. Imsize translates the pixel coordinates so the image centre is 0,0.


def DICAdjust(K,X,Y,imsize):
    
    #Change image coordinate system so centre is (0,0)
    Xcentred=X-imsize/2
    Ycentred=Y-imsize/2
    
    #Initial Guesses
    Xnew=Xcentred
    Ynew=Ycentred 
    #Set error to initialise loop
    RMSErr=1
    
    while RMSErr>1e-10:
        
        ##Find RMS Error
        XErr = np.sqrt(np.mean( (Xcentred- K*Xnew*(Xnew**2 + Ynew**2) - Xnew)**2))
        YErr = np.sqrt(np.mean((Ycentred- K*Ynew*(Xnew**2 + Ynew**2) - Ynew)**2))
        
        # Update Guesses, dont change (X,Y)Centred,
        # as the distorted coordinates are unchanged in the equation 
        # y= ydist - K1*y(x^2+y^2)
        Xnew = Xcentred- K*Xnew*(Xnew**2 + Ynew**2)
        Ynew = Ycentred- K*Ynew*(Xnew**2 + Ynew**2)
        # Calculate RMS Error, to check for convergance
        RMSErr=max(XErr,YErr)
    
    #Recentre image in reference
    Xnew+=imsize/2
    Ynew+=imsize/2
    
    return [Xnew,Ynew]

Here are the .XML parameters I used. I tried to keep them as simple as possible to eliminate other issues.

XML


	<ParameterList name="CAMERA 0">
    <Parameter name="CAMERA_ID" type="string" value="Camera A" />
    <Parameter name="CX" type="double" value="1024"/>
    <Parameter name="CY" type="double" value="1024" />
    <Parameter name="FX" type="double" value="1" />
    <Parameter name="FY" type="double" value="1" />
    <Parameter name="FS" type="double" value="0" />
    <Parameter name="K1" type="double" value="7.829630306643459e-09" />
    <Parameter name="LENS_DISTORTION_MODEL" type="string" value="K1R3_K2R5_K3R7" />
    <Parameter name="ALPHA" type="double" value="0" />
    <Parameter name="BETA" type="double" value="0" />
    <Parameter name="GAMMA" type="double" value="0" />
    <Parameter name="TX" type="double" value="0" />
    <Parameter name="TY" type="double" value="0" />
    <Parameter name="TZ" type="double" value="1" />
    <Parameter name="IMAGE_HEIGHT_WIDTH" type="string" value="{2048, 2048}" />
    <Parameter name="PIXEL_DEPTH" type="int" value="8" />
  </ParameterList>

I can forward the images I used if you would like.

Jul 20 '21 01:07 oddroj

Thanks for the detailed information regarding the issues you've uncovered. After looking into this I have a few things to share (some of which will likely be disappointing...):

There hasn't been much testing of using a distortion model in general in DICe. It appears that in several places, DICe assumes the user is using the OpenCV distortion model (which has even powered radial terms, not odd powered like the paper you reference above). This needs to be protected against by verifying the distortion model (this is a bug, so I'm adding the bug label to this issue).
Secondly, and this is the disappointing part, it also appears that lens distortions are not automatically taken into account when the user supplies a camera system file with k1/k2 or even if the user specifies a lens distortion model. There is another parameter that needs to be set in the input to undistort the images, but even if you add that parameter manually the distortion model is hard coded to the OpenCV one. (this is another bug).
The enhancement label is just to note that people would like to use more than the OpenCV distortion model so we need to enable new features.

I'll leave this issue open for the next round of updates to DICe. It would obviously be valuable to have the distortion models working correctly in DICe, I'm just not sure when we are going to get to it. It could be a few months. I will update this issue as soon as these bugs are addressed.

Jul 20 '21 15:07 dicengine

Thanks for your help, it is a little disappointing, but alas it is free software so how can I complain! I think I will just change my distortion model to the one used by OpenCV - that's probably the easiest fix - especially if it's just the exponent on the R. If possible could you forward a note about what parameter needs to be set to undistort the images?

Cheers Jordan

Jul 20 '21 20:07 oddroj

Taking another look at how lens distortions are applied, I found an issue that may have been causing problems. The fix for this is in a commit I just pushed: https://github.com/dicengine/dice/commit/1a0115a21a5c27af8505bda124d40665223f15aa

I will rebuild the installers with this commit included. Can you try with the new version and see if it fixes your issue? I'll add a note when the installers are ready.

Jul 22 '21 21:07 dicengine

The new installers are up with the fix: https://github.com/dicengine/dice/releases/tag/v3.0-beta.3

Jul 22 '21 22:07 dicengine

The new installers are up with the fix: v3.0-beta.3 (release)

Absolutely beautiful work. You've made my day!

Jul 22 '21 23:07 oddroj

K1R3-K2R5-K3R7 Distortion Model - Odd results.