LLFF scenes and NDC coordinates?

[ricshaw]

Does this repo support NDC coordinates for frontal scenes like the LLFF dataset?

[liruilong940607]

Hi I didn't try but I don't see any reason that nerfacc does not work with NDC space. It is just your rays_o and rays_d need to be in NDC space and then you can just use nerfacc to do marching and rendering as usual.

[ricshaw]

Hey thanks for the reply. I have now implemented the LLFF dataloader with examples/train_mlp_nerf.py, but unfortunately I am getting slightly worse results than vanilla nerf or TensoRF. On the fern scene I get an average PSNR=23.42dB, but the original nerf paper reports 25.17dB and with TensoRF I get 23.91dB. I am using aabb=-1.5,-1.67,-1.0,1.5,1.67,1.0 with render_n_samples=2048 and grid_resolution=128, trained for 100,000 steps. Not quite sure why it is performing worse...

[AIBluefisher]

We can find a discussion here for InstantNGP of how to use the LLFF dataset.

[97littleleaf11]

@ricshaw could you please share your branch with the LLFF dataloader?

[97littleleaf11]

Hey thanks for the reply. I have now implemented the LLFF dataloader with examples/train_mlp_nerf.py, but unfortunately I am getting slightly worse results than vanilla nerf or TensoRF. On the fern scene I get an average PSNR=23.42dB, but the original nerf paper reports 25.17dB and with TensoRF I get 23.91dB. I am using aabb=-1.5,-1.67,-1.0,1.5,1.67,1.0 with render_n_samples=2048 and grid_resolution=128, trained for 100,000 steps. Not quite sure why it is performing worse...

The difference between NDC and scene contraction from mipnerf-360 might be the course of worse performance? I also tried to run the LLFF dataset using train_mlp_nerf and can confirm it generates worse results. https://github.com/NVlabs/instant-ngp/discussions/367#discussioncomment-3205524 ngp-pl without NDC also reported a worse result.