nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

Poisson mesh issue with normal estimation

Open HungNgoCT opened this issue 2 years ago • 1 comments

Hi there,

I am using nerf to export mesh using Poisson. However, the result was like this video below.

I predict that the issue due to normal estimation problem when training with normal prediction. I render "normal show" on and off in Meshlab for reference.

Can anyone give me an advice on this to improve mesh generation?

Thanks in advance.

https://user-images.githubusercontent.com/110378166/218024760-55561978-d5d8-494f-bd1e-0197ec9acb91.mp4

HungNgoCT avatar Feb 10 '23 07:02 HungNgoCT

The mesh generation method is fairly simplistic at the moment so results like this aren't that unprecedented. If you want a better mesh, consider exporting at a point cloud and using other tools to convert the pointcloud into a mesh. Alternatively look into sdfstudio which is better designed to recover geometry.

tancik avatar Feb 10 '23 17:02 tancik

The mesh generation method is fairly simplistic at the moment so results like this aren't that unprecedented. If you want a better mesh, consider exporting at a point cloud and using other tools to convert the pointcloud into a mesh. Alternatively look into sdfstudio which is better designed to recover geometry.

Thank you @tancik for helpful advice. The problem as above due to not good colmap estimation, I think. After remove some not good images, I can get reasonable result now. However, when I increased the number of faces, some "noise" faces appear while small number of face did not. Also, when increasing the value std-ratio also reduce these "noise". Please see the video here.

https://user-images.githubusercontent.com/110378166/222665413-66d81f96-fca3-470a-83a2-9d452a23bcd3.mp4

You can see that the "noise" faces have very different color with surrounding while they locate on a similar surface with surrounding. My question is: why the colors of the "noise" faces are very different because I understood that colors are predicted by nerf based on location? Do you have any advice on this?

HungNgoCT avatar Mar 03 '23 08:03 HungNgoCT

I'm not really sure what is going on. The colors are predicted by casting small around the surface of the object along the normals. These rays might have poor normals, or they might be cast in an area with bad geometry.

tancik avatar Mar 03 '23 17:03 tancik