pytorch3d

pytorch3d copied to clipboard

🚀 Feature

Reimplement/Port the Differentiable Poisson Surface Reconstruction algorithm from the paper Shape As Points: A Differentiable Poisson Solver into PyTorch3D. This implementation will provide mesh reconstruction functionality from point clouds in a differentiable manner.

NOTE: I have reviewed the existing list of Issues tagged with the label 'enhancement` and found some already existing requests for similar features; however, this specific method was not mentioned. Also, the paper is under the MIT license.

Motivation

Similar to the mentions in Issues #331 and #136, Poisson Surface Reconstruction has been adopted by a lot of papers to reconstruct meshes from point clouds, and doing so while admitting gradients would fit PyTorch3D's use cases for 3D deep learning.

Pitch

I would like to contribute by porting this algorithm to PyTorch3D through integrating the existing implementation in the paper with PyTorch3D structures and other ops. My implementation would replace functions in the paper like point rasterization that rasterizes points to a voxel grid with PyTorch3D's Point Cloud to Volume ops or it would directly operate on Volumes, and return a field that can be later used with PyTorch3D marching cubes operation.

I would be happy to contribute this feature if it seems like a good path to follow and if there are no policies against porting external sources (I would also greatly appreciate any suggestions in designing the roadmap and API structure 😄).