camera.sdf: redundancy between //distortion/center and //lens/intrinsics/c* should be fixed

[scpeters]

Desired behavior

The //camera/distortion/center and //camera/lens/intrinsics/cx & //camera/lens/intrinsics/cy parameters describe the same information about the optical center of an image. This can lead to confusion for users and requires duplication of parameter values in order to maintain consistency.

We could consider deprecating one of these parameters and exclusively using the other.

Alternatives considered

Implementation suggestion

Additional context

[deepanshubansal01]

I might be wrong but it looks like the optical centre and distortion centre are two different things and we cannot deprecate either of them https://developer.apple.com/documentation/avfoundation/avcameracalibrationdata/2881131-lensdistortioncenter

I also went through gazebo code and it looks like optical centre and distortion centre do not depend on each other.

The centre tag of distortion tag takes normalised values and the image is distorted based on distortion centre https://github.com/osrf/gazebo/blob/127108a4137c70e59704281399465efe0e90fc19/gazebo/rendering/Distortion.hh#L113-L129

The wiki page also has some reference of how distortion is applied to an image https://en.wikipedia.org/wiki/Distortion_%28optics%29#Software_correction

@adityapande-1995 @jacobperron @scpeters

[adityapande-1995]

@deepanshubansal01 Can we have a small example world file to demonstrate that intrinsic centres and distortion centres can be different ?

[deepanshubansal01]

@deepanshubansal01 Can we have a small example world file to demonstrate that intrinsic centres and distortion centres can be different ?

Sounds good.

[deepanshubansal01]

Based on further testing it seems like the distortion centre directly affects the distorted image and is independent of optical centre i.e the optical centre and distortion centre may or may not be the same.

Here are some visuals:

Undistorted Image:

Distorted Image with centre tag as 0.5, 0.5

Distorted Image with centre tag as 0.6, 0.6

[mallanmba]

When you run a calibration on a real world camera you get one "center", the principal point (i.e. cx, cy in the intrinsics matrix). I'm pretty sure that should be used as the distortion center - but we should get a computer vision expert to weigh in on this. Perhaps the distortion center should be initialized to cx and cy unless a user specifically overrides it, and the documentation about the tags clarified.

[kjeppesen1]

When you run a calibration on a real world camera you get one "center", the principal point (i.e. cx, cy in the intrinsics matrix). I'm pretty sure that should be used as the distortion center - but we should get a computer vision expert to weigh in on this. Perhaps the distortion center should be initialized to cx and cy unless a user specifically overrides it, and the documentation about the tags clarified.

I concur with @mallanmba regarding that in a real camera calibration you get one center (cx, cy) as part of the camera intrinsic matrix, and separately you get the distortion coefficients (k1/k2/k3 and p1/p2). See OpenCV's calibrateCamera() function for an example. I think that it is implied that the coefficients that are calculated in a camera calibration correspond to the same intrinsic matrix cx/cy, but I suppose there might be some users who might want to have those be separate for some specialized reason? I also agree that we should have a CV expert validate this though.

[deepanshubansal01]

After our discussion offline I got a better understanding of the problem we were facing, and I agree now that since we only get one cx, cy after camera calibration that's supposed to be equal to the distorted centre since the image may be distorted during calibration. I might have got confused b/w undistorted optical centre with distorted centre earlier.

The distortion matrix might come in handy when we want to get the undistorted image back from original raw distorted image for which the camera has been calibrated originally.

The camera info msg also explicitly mentions that K matrix is for the raw distorted image http://docs.ros.org/en/api/sensor_msgs/html/msg/CameraInfo.html