plans to add meshing?

[agporto]

Thanks for this great repo. Performance looks really great. Are there any plans to add reconstruction of high resolution meshes as a final step?

[jytime]

Hi Arthur,

Thanks for the suggestion! We haven’t planned for high-resolution mesh reconstruction yet, but I can discuss it with my coauthors. Could you share your thoughts on recent meshing methods? Are they too slow, inaccurate, or do they have other limitations that concern you? Understanding the key challenges would help us explore potential improvements.

[agporto]

Thanks for the quick response. I’ve noticed that while traditional meshing methods (e.g. Poisson reconstruction or Marching cubes) can generate reasonably accurate geometry, the overall pipeline often slows down when it comes to high-resolution reconstructions. In particular, generating and optimizing the UV mapping for these meshes is a key bottleneck. Creating a seamless, distortion-free UV layout for complex, high-resolution surfaces can be computationally intensive, especially when you need to blend textures from multiple views and minimize seams. The challenge isn’t just in the mesh extraction but in ensuring that the texture mapping accurately reflects the original images without artifacts. I’d be interested in hearing if you’ve seen any promising approaches to streamline this aspect of the process.

[oelmekki]

By the way, not as advanced than what Arthur is suggesting, but being able to use vggt to generate images for UV mapping would already be quite awesome.

[destin-v]

Generating high quality meshes in .glb, .obj, or .usd formats would make it much easier to work with using tools like Blender3D. Furthermore, if you added mesh generation users can use it in robotic simulators like Isaac Lab. I think having the ability to export as a mesh is very important if you want to work with interactive environments.

[fankeyu123]

Generating high quality meshes in .glb, .obj, or .usd formats would make it much easier to work with using tools like Blender3D. Furthermore, if you added mesh generation users can use it in robotic simulators like Isaac Lab. I think having the ability to export as a mesh is very important if you want to work with interactive environments.

IF you want to use the asset in robotic simulators, How about the physics parameters like mass/friction?

[asrays]

@agporto @jytime @destin-v hey is there any way to convert generated .glb file into .pcd format without losing color information. We can then run surface reconstruction very easily? Please help ..

[destin-v]

@asrays that’s an interesting question. Each file format has its own way of handling color and texture. That has resulted in many different formats which do not necessarily translate well into other formats.

To take Blender for example, if you use Principals BSDF, it has the ability to export that coloring out to other formats. But if you do formatting on it via Geometry Nodes then it wont export to other formats. In order to understand whether something can be exported you have to look at the specific way coloring or texture is encoded and whether that can be exported into the target format.

[Chinafsh]

Thanks for the quick response. I’ve noticed that while traditional meshing methods (e.g. Poisson reconstruction or Marching cubes) can generate reasonably accurate geometry, the overall pipeline often slows down when it comes to high-resolution reconstructions. In particular, generating and optimizing the UV mapping for these meshes is a key bottleneck. Creating a seamless, distortion-free UV layout for complex, high-resolution surfaces can be computationally intensive, especially when you need to blend textures from multiple views and minimize seams. The challenge isn’t just in the mesh extraction but in ensuring that the texture mapping accurately reflects the original images without artifacts. I’d be interested in hearing if you’ve seen any promising approaches to streamline this aspect of the process.

I know there is one technical paper could solve ur concerns. TwinTex is all you need.

[Mert-Eren-Karabulut]

Hi Arthur,

Thanks for the suggestion! We haven’t planned for high-resolution mesh reconstruction yet, but I can discuss it with my coauthors. Could you share your thoughts on recent meshing methods? Are they too slow, inaccurate, or do they have other limitations that concern you? Understanding the key challenges would help us explore potential improvements.

Hey, first of all thanks for the great work you guys done here.

I too think that coupling this work with a good meshing pipeline is the key to make this work a true art. In the past I have worked with a openMVG + openMVS pipeline, back in those days the real struggle was to get a good image alignment in short amount of time. But the SfM was always taking so long especially with high res images and sequential pipelines. (which always gave the best result).

But you basically resolved the SfM part with this work :D (which was the real time consumer)

I have always been grateful of the meshes coming out of the openMVS after the sfm is done and camera poses are generated. I strongly advise you to take a look at openMVS for meshing. It is written in C++, has CUDA support for most of their pipeline and under correct settings it is blazing fast. I would love to see a combination of VGGT and openMVS. It can work wonders.