How to optimize EmerNeRF

[nevergone123]

Thanks for your wonderful works!

I'm doing a project related to AD simulation. I ran emernerf on scene in which ego car move at a medium or high speed, like waymo scene id 2. But the decomposition result is not good.

Following the approach of DynIBAR, I tried to improve emernerf by adding 2D optical flow supervision to the training process. I merged the flow loss into the total pixel loss. However, the flow loss did not converge during training, and the overall training results became worse.

So the questions are:

1. How can I optimize emernerf? For example, improve its decomposition ability.
2. If I use additional 2D optical flow supervision, besides merging the flow loss into the total_pixel_loss, what else should I do? Will this approach improve the decomposition results?

Looking forward to your advice. Thank you once again!

[ntaquan0125]

I think outliers come from lighting variations between views. And 2D flow supervision for fast ego-motion data should be handled with care.