bmild
Some question about the reconstruction on the reflecting region of the object
Hi, thank you for your nice work! I have tried your Nerf with our own scenes(we have taken 50 pictures), and I found the generated video is not satisfactory. I found there is some blur like cloud above the object in the generated video. I think it may be the reconstrution part of the reflecting region of the object, I am wondering how to avoid this situation? Thank you!
It cannot generate good result on the top part of the object because training images don't contain much of that region. One thing you can try is to modify the rendering space, especially try to rotate the virtual camera downwards until the noisy cloud is not visible. I think the relevant code is here: https://github.com/bmild/nerf/blob/8edde335d2b18188769850b03c45515352d66b31/load_llff.py#L202-L204 Actually I really don't like this part because it only gives some magic numbers without explaining what does what, you might need to decipher by yourself; I have somewhat detailed comments in my implementation based on my understanding, you can check.
For a little bit of context @kwea123 , that part of the code just creates a world coordinate system for the scene such that the object is at the center with the z axis pointing upward. (The previous line calling min_line_dist finds the "focus point" of the images by calculating the point in 3d space that is nearest to all the camera axes.) All that [.1,.2,.3] is doing is allowing us to create an x and y axis to go with the z axis, using the cross product to make sure all 3 vectors are perpendicular. It's completely arbitrary and could be replaced by a random 3 vector, but we just use a hardcoded constant to make it reproducible.
Yes, so regarding the question of the op, to remove the noisy cloud in the synthesized image, one should change this 0.1, 0.2, 0.3 vector to make the camera point more downwards, but it can only be done by trial and error....
Thank you for your advice! I took a new 360 degree shot of the side of the object, the noisy cloud problem has been reduced. But there are still some blurs in the video boundaries. Is this caused by background things?
