introlab
Texturing untextured surfaces with nearby Colors within the iOS App
The vertex mesh assembling mode does this but unfortunately the textured parts are extremely low quality, and as you may know, not having textures on the model itself makes it incompatible with basic software.
Also, if you add something like this, please keep the mesh enclosing feature ( hole filling functionality ) available, RTABMap is the only iOS software that provides actual complete 360 models, which even closes bigger voids in the capture that the LiDAR Scanner didn't reach !
It would be amazing to have a setting that can maximize the hole filling algorithm strength in the future as well, knowing that it can still leave some big gaps open.
That combined with the requested feature in this post, additionally texturing meshes based on nearby colors, would finally enable full environmental scanning directly from the iPhone !
It would finally allow revisiting complete captures in VR, without having colorless or big voids in the models !
Thanks for the suggestion. I'll check we can do that for next ios version release.
Edit: this should be already possible on Desktop app with multiband texturing enabled.
Yup, using the PC to export a scan from a standalone device ( iOS ) to another standalone device ( Meta Quest VR ) isn't feasible.
This feature would be a game changer for sure! The app is already a powerhouse. Hats off to @matlabbe 🙏
To add to the discussion, it seems that we can add vertex color in OBJ format (ref: https://paulbourke.net/dataformats/obj/colour.html and https://en.wikipedia.org/wiki/Wavefront_.obj_file#Geometric_vertex), though it is not standard, so not sure about the support in software you may use (see also https://blender.stackexchange.com/questions/279182/how-can-i-import-an-obj-with-vertex-colors-into-blender-3-2).
If it works, that would be the easiest/fastest way to do it, as we should already know the closest color of each vertex in the mesh before texturing it.
Based on Gemini:
Yes, an OBJ file can contain both vertex colors and texture information simultaneously, allowing you to use both methods to color a 3D model within a compatible software depending on your needs; essentially, each vertex can have its own color assigned by vertex color data while also referencing a texture map for additional detail.
Unfortunately , PointCloudLibrary doesn't support color vertex for OBJ, it only writes XYZ values: https://github.com/PointCloudLibrary/pcl/blob/1590ba5f6b8738d308fca8657dcec509f4537225/io/src/obj_io.cpp#L1177-L1198
That means we need another way/library to export to OBJ, or modify the one of the PointCloudLibrary, see pcl::io::saveOBJFile().
The new version with this feature has been released to App Store.
You can enable this export mode by changing the following settings under Settings->Assembling:
This makes the mesh fully closed:
Note that there are two modes on how we attribute vertex colors:
By clicking on the Texture/Color blend visualization option, we better see the difference:
Depending on the software you are using, this is how the meshes may look like. To get the previous image with no seams, we have a custom shader to not apply vertex color when there is a texture.
When we export to OBJ, if we enabled vertex color, the color will be added to each vertex.
Thank you so much!
Are the filled-in holes colored with textures or vertex coloring in an extracted model?
The filled-in holes are colored from the vertex color. They would appear white if we keep "Colorless" texture vertex color policy.