About Unit Sphere used in get_multiview_rays

[saulgooodman]

Hello, I would like to understand the approach of the get_multiview_rays function when running run_mv.py. Overall, it seems that the scene is confined within a large sphere (for example, the sphere limit radius for the ScanNet dataset is set to 9 in the config), and the RayDF uses the points hit by the training rays as the center to draw a unit circle. Multi-view rays are then obtained by random sampling on this unit sphere, and a magnificent multi-view consistency depth loss is used at the end. I'm not sure if my understanding is correct? Additionally, I have some questions about the setting of the unit circle size. For instance, in the config for the Blender dataset, it shows radius=1.5. What if the training rays hit a point near the edge, and then a unit circle is drawn with this point as the center? Wouldn't this exceed the boundary set by the radius? I wonder what impact this might have on the training, especially when customizing a dataset. If the bounding sphere of the dataset is particularly small, would using a unit circle as the center affect the training? Thank you! By the way I drew a simple graph to illustrate my frustration. Sorry for the bad hand writing and drawing.(￣﹏￣)

[zhuomanliu]

Since a spherical parameterization will be applied to all rays before training, including the sampled multi-view rays, it will have no effect even if the sampled ray 'exceeds' the scene_bounding sphere. (please refer to Figure 4 in the paper, we draw an 'x' mark to show the starting point of each multi-view ray.)