Stereo camera parameter descriptions missing/unclear

[rokusottervanger]

The stereo camera sensor model includes the following parameters:

• p_1
• p_2
• p_3
• p_4
• p_5
• lateral_factor
• depth_to_disparity_factor

However, there is no explanation of what these parameters mean. Can anyone help me with this?

[Mayankm96]

Any updates on this? I am trying to use Zed camera for elevation mapping and am stuck at this step as well.

[peci1]

See https://ieeexplore.ieee.org/document/6375037/ .

[maximilianwulf]

Hi @rokusottervanger, this Ph.D. thesis describes the parameters in more detail compared to the paper.

In the future, we will add a detailed description of the parameters.

Regarding new camera sensors, the users have to measure the corresponding parameters by themselves.

[MattDom]

Thank you for this additional source, @maximilianwulf.

Unfortunately, I don't fully understand the parameters of the sensor model, yet. I've "reverse-engineered" (in other words, looked it here up) this particular formula:

However, I asume, that the depth-to-disparity-factor is given by the focal length (f) times the baseline (B). Further, I denoted the disparity by d. J(i) is the column and I(i) the row, as it seems to me.

Because I didn't know what to do further, I plotted everthing with the provided parameters in here. For the disparity, I asumend a range from 30 to 180 (just for a first shot).

So, at first glance, it seems plausible so far. The devation increases with larger distances, as expected.

The questions that remain:

• How to solve for p1 to p5? I could imagine an iterative solution, started with an educated guess...
• Where does the 240 comes from? (It seems a bit magic, or does it refer to a specific camera resolution?)
• In the example config file a related project work is referenced. I searched quite a long time, but couldn't find anything. It would be great if someone could send me this work.

[MattDom]

Okay, here are some news! At first, forget what I've written in the post above.

So, let's have a look at new theory. The first mistake in the above post was not to respect the spatiality of an image. The variance is not constant over the whole image. Why am I thinking that? Have a look at the following image. The surface in front of the camera is perfectly perpendicular the to optical axis. So, the two points have an equal distance to the camera in z-direction. But, the euclidean distance is unequal (D1<D2). For my understanding, this means a point that is farther from the camera is more prone to errors.

This theory fits pretty well to the formula above. I used these parameters for testing and generated an image with the dimension 640x480 and a constant disparity of 100 (just a first-shot).

Here is the result:

Looks good at first glace. However, it doesn't tell us what are the parameters stand for. And last but not leat where this magic 240 comes from.

The magic 240 was my biggest question. I'm pretty sure that this is half the resolution in one direction (2*240=480 --> a common resolution). But why is it fixed in the source code?

Next, the parameters. At first p_1 seems to be just a common offset for the whole variance all over the image. p_2 looks like a gain without any (direct) relation to the disparity and p_5 is the same with respect to the disparity?

p_1 =   0.03287; % Offset without depending on disparity?
p_2 =  -0.0001276; % common gain?
p_3 =   0.4850; % Kind of a shift?
p_4 =   320; % 399.1046; % vertical shift?
p_5 =   0.000006735; % disparity related gain?

p_4 can be a vertical shift or, in other words, the principal point? I adjusted it in to my image resoltion and it seems to represent the principal point. So, just p_3 is left. This seems to be bit tricky. It shifts the image horizontally with respect to the disparity. Have a look at the image below.

But now, two questions are left. How to setup the measurement and select the parameters p_1, p_2, p_3 and p_5?

Every idea is welcome!

[arainbilal]

Hi @maximilianwulf

@pfankhauser thesis you have referenced in this post referred to a student project report for more details about the sensor error model for the RealSense ZR300 on page 35, as follows: Takahiro, M., Fankhauser, P., and Bjelonic, M. (2017). “Error Analysis of the RealSense ZR300, Obstacle Removal for Elevation Mapping, and Swing Trajectory Planning for ANYmal”. Student’s Project. ETH Zurich, Switzerland. Unfortunately, this report is not accessible; I think should be accessible from ETH library. Not sure if @MattDom has found an answer about what these parameters mean, looking for more details too. If possible please help. Thanks.

[MattDom]

I found an interesting repo, submitted by @HannesKeller. It contains MATLAB code to generate the sensor model for a stereo camera, based on these mystic parameters. Actually, I don't have that much time to analyze the code in depth...But at first glance, it should be possible to work this out.

And maybe Hannes will read this and share his work, mentioned in the code :smiley:

[maximilianwulf]

@MattDom thank you for your input and the link to the repo. @HannesKeller seems to be the original author of that sensor processor.

And maybe Hannes will read this and share his work, mentioned in the code smiley.

Let's see. :smiley:

@arainbilal does this link work for you?

[arainbilal]

Thanks @maximilianwulf. Yes it does. Although, I end up adding another sensor processor with in the elevation mapping framework using different parameters from my own tests, but it is interesting to review the work done by Nguyen et.al and then extended by @pfankhauser. Thanks.

[maximilianwulf]

@arainbilal good to know. In what setup are you using elevation mapping?

[arainbilal]

For terrain mapping in robotics applications and only for research purpose.

[maximilianwulf]

@arainbilal thanks, I was just curious on what cool projects elevation map gets used.

[xiandaoo]

@maximilianwulf @MattDom I ran into a similar problem, and luckily I seemed to find the answer in an article. However, it is not known whether this error model is completely correct. Hope this answer can help those in need. article

[maximilianwulf]

Thank you @xiandaoo. I guess it still misses p_0, p_1, p_2, p_3, p_4, p_5.

From a practical standpoint, I think tuning these parameters is not necessary.