nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

conflict between scene_box aabb with disable_scene_contraction in instant-ngp / OccGridEstimator

Open hoanhle opened this issue 6 months ago • 4 comments

OccGridEstimator uses different levels of aabbs to know where to sample the points from with difference levels of details

        # multiple levels of aabbs
        aabbs = torch.stack(
            [_enlarge_aabb(roi_aabb, 2**i) for i in range(levels)], dim=0
        )
...
  intervals, samples = traverse_grids(
            rays_o,
            rays_d,
            self.binaries,
            self.aabbs,
            near_planes=near_planes,
            far_planes=far_planes,
            step_size=render_step_size,
            cone_angle=cone_angle,
        )
        t_starts = intervals.vals[intervals.is_left]
        t_ends = intervals.vals[intervals.is_right]

But those samples outside the original aabb is not used anyway if we disable the scene contraction

        if self.spatial_distortion is not None:
            positions = ray_samples.frustums.get_positions()
            positions = self.spatial_distortion(positions)
            positions = (positions + 2.0) / 4.0
        else:
            positions = SceneBox.get_normalized_positions(ray_samples.frustums.get_positions(), self.aabb)

  # Make sure the tcnn gets inputs between 0 and 1.
        selector = ((positions > 0.0) & (positions < 1.0)).all(dim=-1)
        positions = positions * selector[..., None]

What am I missing here? Shouldn't we instead do something like _enlarge_aabb with factor 0.5 when the scene contraction is disabled?

hoanhle avatar Sep 02 '24 13:09 hoanhle