arcore-depth-lab
arcore-depth-lab copied to clipboard
googlesamples
Continuous evironment mesh building
Hello, first of all - thank you for such a great project!
I have a question about continuous evironment mesh building - could you point me a way how to implement creating enviroment "scanner", i want to create approximate model of around environment - i see this like every new frame will fill and deform already existing "scanned" mesh and make it more accurate (in future - with environment texture mapping). I think the simpliest way is to subtract new scanned mesh from already scanned.
Maybe it can be a good and very interesting objective for a new sample!)
Yes, I agree that an interesting objective is to reimplement this! My pointers are the KinectFusion papers and follow up works:
- https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/kinectfusion-uist-comp.pdf
- https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ismar2011.pdf
- https://www.youtube.com/watch?v=KOUSSlKUJ-A
The current version of DepthLab does not support continuous reconstruction yet, but there is potential. You can also see this https://youtu.be/EerbBxLt7hI?t=48 that leveraged our raw Depth API as a partner.
hello @bitk0der i am doing something similar to what you describe, could we talk on discord and maybe help each other? ademord#2999
@Ademord Sent you a friend request on discord, I am looking at doing something similar. I have found info on using Open3d libraries to construct meshes, but I lack the knowledge to integrate it with Unity. I don't necessarily need mesh generation to be continuous and real time, but I would need to still generate meshes "in-game". My thought process was that I could save the point cloud, generate a mesh, and allow the user to manipulate the mesh.
I'll close it for now until I have further update in continuous mesh, btw, if you are interested in 3D photos with depth mesh, check out the demo in https://visualblocks.withgoogle.com/#/edit/_?project_json=https:%2F%2Fraw.githubusercontent.com%2Fgoogle%2Fvisualblocks%2Fmain%2Fpipelines%2Fgraphics%2F3dphoto_portrait_depth.json
more demos in https://github.com/google/visualblocks/tree/main/pipelines