instant-ngp
instant-ngp copied to clipboard
NVlabs
export pointcloud
Hello,
Is there any way to export a pointcloud (ideally with colors), rather than using the Marching Cubes mesh exporter built into the application ?
The mesh produced with this method is not great, and it is probably easier to clean up stray points, then create a mesh in other applications
thank you
Andrew
At the moment you can export a 3D density grid and threshold it to get a pointcloud
The method Testbed::compute_and_save_png_slices generates a grayscale image with density slices, that can be read like below in Python
from skimage.util import view_as_blocks
grid_3d = view_as_blocks(image, (res[1], res[0]))/255.0
grid_3d = grid_3d.reshape(-1, res[1], res[0])[:res[2], :, :].T
You can then threshold it
vertices = np.array(np.where(grid_3d > threshold)).T.astype(float)
It's only a pointcloud of 3D coordinates inside the axis aligned bounding box that was used to render the density slices (unit box by default).
If you want to specify your own rendering bbox by passing aabb to compute_and_save_png_slices,don't forget that it has to be expressed in the local NGP coordinates, not the same as the NeRF convention. Have a look in nerf_loader.h (nerf_matrix_to_ngp)
@Tom94 A pointcloud with a good spatial resolution requires way too much VRAM to store the entire density 3D grid Since the scene is almost always sparse, we should rather directly work with an internal sparse 3D structure, and the value of density doesn't really matter if we threshold it afterwards to generate vertices
@barney2074 The output_render.mp4 that is generated for my custom example is good, but the mesh .obj is very bad. I can barely tell the object is in the obj file. I trained on the cloud for 20000 steps with transforms.json and another 20000 with the base_cam.json and the 20000.msgpack. Kindly advice what I can do to get a proper mesh output.
Hello,
Is there any way to export a pointcloud (ideally with colors), rather than using the Marching Cubes mesh exporter built into the application ?
The mesh produced with this method is not great, and it is probably easier to clean up stray points, then create a mesh in other applications
thank you
Andrew
Hi Andrew, have you been able to export Point Cloud out of Instant NGP?
Hi @mgupta70
No- but I haven't tried since the above post. I believe (although I haven't tried) other applications such as LumaAI or https://docs.nerf.studio/en/latest/quickstart/export_geometry.html can export pointclouds.
https://www.youtube.com/watch?v=h5EWiRRxYEQ
Andrew