Deformation for view direction?

[fishfishson]

Hi, author!

Both nerfies and non-rigid nerf use deformation field. However, in non-rigid nerf they recompute view direction after deformation. How does nerfies deal with it? I quickly reviewed your implementation and did not find related operations. Looking forward to your reply!

[longbowzhang]

Similar concerns here. Quite curious why not apply deformation to the view directions? Could you explain a little bit? @keunhong Looking forward to you reply

[keunhong]

Hi, sorry for the late response.

This is a very good question. Our method uses observation-frame view directions which remain correct after deformation. With the appearance latent code, this provides a means to somewhat explain view dependent changes. However, you're right that since the scene geometry deforms, the reflected colors should change as well. To truly model view dependence, we'd have to consider local geometry e.g., by using normals to compute reflected ray directions. I'm sure future work will try to do this :)