Define Fisheye camera model with distortion

[raoqiang]

🚀 Feature

api for add custum camera model with distortion like fisheye camera；to simulate the fisheye camera and get the sementic segmenation and depth

Motivation

simulation for large fov camera and simulation the fisheye camera which widely used in industrial instruments

[dhruvbatra]

@Skylion007

[Skylion007]

@raoqiang we actually do support fisheye cameras and have for a while. The python bindings are merged into this PR and you see how to construct the corresponding camera sensor there: https://github.com/facebookresearch/habitat-sim/pull/1168

[raoqiang]

@Skylion007 thanks for your reply, I have tried the double sphere fisheye camera model to generate fisheye corlor image and depth image, but I found that the depth is not correct and I tested the depth to recontructed the depth to cloud, the center part is correct but the edge is wrong

the fragment is the src/shader/doubleSphereCamera.frag the double sphere parameter I used is this "cam.intrinsics": [ { "fx": 312.99588161843817, "fy": 312.9214941782706, "cx": 638.7854720110728, "cy": 514.5354880999439, "xi1": -0.17918626779269324, "xi2": 0.590840604914911 } ],

there is an obviously mistake on the depth image

how can I get the correct depth from fisheye camera model scerely wish for your reply !

[aclegg3]

@Skylion007 any thoughts on this? Is this a bug?

[ZivKidd]

met the same problem in equirectangular depth image, it seems like a cube shaped depth map is added to the depthmap, maybe the cubemap to fisheye and equirectangular transform is wrong