FOV_MASK for FisheEyeProjection is redundant to Fx,Fy.

[Skylion007]

Master

Habitat is under active development, and we advise users to restrict themselves to stable releases of Habitat-Lab and Habitat-Sim. The bug you are about to report may already be fixed in the latest version.

Master branch contains 'bleeding edge' code, but we do appreciate bug reports for it!

🐛 Bug

• The FisheEye Observation Transformer for the FOV mask and the fx/fy focal length params. These seem like they could conflict quite easily and the FOV mask could be computed directly from the other parameters. Any thoughts on why both are included @matsuren ?

[matsuren]

Sorry for the late reply (long holidays in my country).

From my understanding, fov mask and fx/fy focal length both are needed because there are separated parameters. For instance, it's possible to have two circular fisheye cameras with the same fx/fy focal length but different fov_mask. (If you don't consider circular fisheye image, maybe fov mask is not needed)

I guess fx/fy focal length parameters here could be considered as distortion parameters for fisheye camera. (See Eq. (40) in https://arxiv.org/pdf/1807.08957.pdf )

If I miss something, please let me know.

Thank you,

[bigbike]

Hi @matsuren,

Good to see your explanation.

I see. So the fov_mask is to make the image (observation) a circle in your implementation. This is not necessary. Simply changing the focal length (fx, fy) can easily achieve such visual effects. See the attached pictures.

Experiment settings: The canvas (image plane, or camera's internal image sensor) is w x h (assuming h < w). In our case it is 1600 x 1200 (pixels). We assume fx = fy = f; The lens is BF2M2020S23. Double sphere camera parameters can be found in the paper.

• In pic 1, we set f = h / 2.0. The image circle (https://en.wikipedia.org/wiki/Image_circle#/media/File:Image_circle.jpg) is now larger than our canvas. So what you see in the picture is just a rectangle subset. (NO black pixels.)

• In pic 2, we set f = h / 3.3. The image circle is not big enough to cover our canvas. It produces a similar image as the paper does (1st pic in page 1: https://arxiv.org/pdf/1807.08957.pdf). It has 4 dark corners where the lens does not cover the sensor, which is called "vignetting". (https://en.wikipedia.org/wiki/Vignetting)

• In pic 3, we set f = h / 6.0. In this case the image circle is much smaller than our canvas, and thus it falls completely onto the canvas. It has black circle around it.

Normally, vignetting is often an unintended and undesired effect caused by camera settings or lens limitations. It happens when the lens format is smaller than the image sensor format. For example you put a lens designed for APS-C camera to an Full-frame camera. (https://en.wikipedia.org/wiki/Full-frame_DSLR#/media/File:Dslr_sensor_comparison.jpg)

[matsuren]

Hi @bigbike, thank you for explanation with images.

If you want to just control distortion effect, only fx/fy are enough as you explained. But I think it's better to have a control of vignetting (black area) to simulate real fisheye images.

The following image is a fisheye image I took before. I used 180 deg fisheye lens and I want to get 180 deg fov. So, the vignetting in this image is desired and intended, otherwise I won't get 180 deg fov.

Thank you,