nerfstudio
nerfstudio copied to clipboard
Add Support for Equirectangular Images
This PR adds support for equirectangular images. The ray directions should be correctly created based on the image type, but the pixel sampler needs to be manually set by a CLI config. Since the cameras in a dataset can technically have multiple types (e.g. equirectangular and perspective), I could not think of a nice way to deal with everything at the same time. In the end, the EquirectangularPixelSampler
isn't really needed, since the inverse CDF acos(1 - 2x)
for mapping to a sphere is already so similar to that of the uniform distribution.
Special thanks for @TheQuantumFractal for initial steps with this PR :)
Things not done yet:
- [x] Equirect Pixel sampler
Things for a later PR:
- Handle masks with equirectangular images.
@kerrj Can I have some help cleaning up the process data script to pass linting? Since you're the original author.
OK, I have reverted all changes to scripts/process_data.py
and kept just the equirect image contributions. The old code is put into the branch https://github.com/nerfstudio-project/nerfstudio/tree/wip/opensfm.
I will actually just finish off the remaining task for this PR before opening it back up for review.