nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

Add Support for Equirectangular Images

Open kevinddchen opened this issue 2 years ago • 2 comments

This PR adds support for equirectangular images. The ray directions should be correctly created based on the image type, but the pixel sampler needs to be manually set by a CLI config. Since the cameras in a dataset can technically have multiple types (e.g. equirectangular and perspective), I could not think of a nice way to deal with everything at the same time. In the end, the EquirectangularPixelSampler isn't really needed, since the inverse CDF acos(1 - 2x) for mapping to a sphere is already so similar to that of the uniform distribution.

Special thanks for @TheQuantumFractal for initial steps with this PR :)

Things not done yet:

  • [x] Equirect Pixel sampler

Things for a later PR:

  • Handle masks with equirectangular images.

kevinddchen avatar Nov 16 '22 06:11 kevinddchen

@kerrj Can I have some help cleaning up the process data script to pass linting? Since you're the original author.

kevinddchen avatar Nov 16 '22 06:11 kevinddchen

OK, I have reverted all changes to scripts/process_data.py and kept just the equirect image contributions. The old code is put into the branch https://github.com/nerfstudio-project/nerfstudio/tree/wip/opensfm.

I will actually just finish off the remaining task for this PR before opening it back up for review.

kevinddchen avatar Nov 17 '22 05:11 kevinddchen