Why using two networks( coarse and fine)?

[wzpscott]

In the paper, you optimize two networks for hierarchical sampling. I wonder whether it is possible to use just one network and perform both uniform and importance sampling on it. Can you provide some insight on it?

[kwea123]

You can read their follow-up work mip-nerf where only one network is used. It performs even better than nerf.

[wzpscott]

You can read their follow-up work mip-nerf where only one network is used. It performs even better than nerf.

Thank you for your quick reply. Mip-NeRF states that

This use of conical frustums and IPE features also allows us to reduce NeRF’s two separate “coarse” and “fine” MLPs into a single multiscale MLP ...

It seems to imply that coarse and fine models are necessary for original NeRF, but it's unclear to me how the IPE features relate to sampling strategy and why it allows using a single network. And I'm confused about the motivation of using two networks as it's more intuitive for me to optimize a single network like mip-nerf.

[ChikaYan]

I was confused by this for a while, too. I think the reason behind it is that coarse samples tend to be sparsely located in the space and be far away from the object surface, therefore they would contain lower frequency details, while fine samples would have details with much higher frequency (think it as each coarse sample should cover a larger volume than the fine sample). As illustrated in fig.4 in mip-nerf paper, the positional encoding used by original NeRF does not distinguish such a difference in frequency, therefore two networks need to be used.