Is it possible to only texture a mesh with custom poses logs?

[sonnnaya]

Describe the problem I am using NeuralRecon with ArKit logs in order to generate a mesh and I am interested in texturing that mesh with meshroom. I was investigating a standard pipeline manually and concluded that it always requires to use the StructureFromMotion Node even in case you are using some kind of custom poses and intrinsics logs in camera.sfm as described here.

I am looking for a possibility to use your well-done project for texturing purposes. As I understand, my poses format needs to be transferred to camera.sfm, but I don't really want to use StructureFromMotion Node, because it requires other nodes to be proceed (FeatureExtraction, ImageMatching, FeatureMatching).

Assume I have formatted my poses and intrinsics to camera.sfm, and I added a path to my mesh with Meshing Node. Is it possible to texture mesh produced from other programs with custom poses, without usage of StructureFromMotion Node?

Dataset Here is a link to the data I want to use for texturing - images, poses and intrinsics. It also contains a mesh produced by NeuralRecon in both .ply and .obj formats.

Additional context First of all, I tried a usual pipeline for reconstruction from images just to be sure it works. Than I tried to convert cameras.sfm file to sfm.abc and to put it with a mesh to the Texturing Node But it failed with the next logs: Program called with the following parameters:

• angleHardThreshold = 90
• bestScoreThreshold = 0.1
• correctEV = 0
• downscale = Unknown Type "unsigned int"
• fillHoles = 0
• flipNormals = 0
• forceVisibleByAllVertices = 0
• imagesFolder = ""
• input = "C:/X/Meshroom/resources/stairs/MeshroomCache/ConvertSfMFormat/2712ab48e0c920720908127e75441795ffc5bdca/sfm.abc"
• inputMesh = "C:/X/Meshroom/resources/stairs/MeshroomCache/MeshFiltering/a3edf1bb248d02b87e3d9b52aa0a063cca40000e/mesh.obj"
• multiBandDownscale = Unknown Type "unsigned int"
• multiBandNbContrib = Unknown Type "class std::vector<int,class std::allocator >"
• output = "C:/X/Meshroom/resources/stairs/MeshroomCache/Texturing/f52c12df0da8224f6e236c5618a334d06f0efc90"
• outputTextureFileType = "png"
• padding = Unknown Type "unsigned int"
• processColorspace = "sRGB"
• subdivisionTargetRatio = 0.8
• textureSide = Unknown Type "unsigned int"
• unwrapMethod = "Basic"
• useScore = 1
• useUDIM = 1
• verboseLevel = "info"
• visibilityRemappingMethod = "PullPush"

[11:49:56.490017][info] Load dense point cloud. [11:49:56.684656][info] Found 1 image dimension(s): [11:49:56.684656][info] - [640x480] [11:49:56.898585][info] Overall maximum dimension: [640x480] [11:49:56.899586][info] Load input mesh. [11:49:56.899586][info] Loading mesh from obj file: C:/X/Meshroom/resources/stairs/MeshroomCache/MeshFiltering/a3edf1bb248d02b87e3d9b52aa0a063cca40000e/mesh.obj [11:49:56.954573][info] - # vertices: 46040 - # normals: 0 - # uv coordinates: 0 - # triangles: 91503 [11:49:57.184199][info] Mesh loaded: - #points: 46040 - # triangles: 91503 [11:49:57.184697][info] Convert dense point cloud into ref mesh [11:49:57.186202][fatal] Texturing: Cannot remap visibilities as there is no reference points.

[natowi]

At the moment I only know of ways to use existing poses and depth maps like https://github.com/alicevision/meshroom/issues/1361

As the error message states, the Texturing node requires the reference points from the sfm.

You could align your Neuralrecon Model with a draft mesh https://github.com/alicevision/meshroom/wiki/Draft-Meshing and then change out the input Mesh in Texturing https://github.com/alicevision/meshroom/wiki/Texturing-after-external-re-topology You need to run the OBJ through the MeshDecimate node first to get a vaild obj format that Texturing accepts (or use the export options described for Blender).

I don´t know if NeuralRecon can save individual depthMaps, in that case they could be used instead the ones by Meshroom.

But I agree, it would be great to have the ability to Texture Meshes only with the known camera positions, which would make the node more versatile.

[natowi]

This is the Result of the textured NeuralRecon Mesh after a rough alignment to the Meshroom draft Mesh (I only used 9 images):

(The Texture quality could be better, but my model alignment was done manually and pretty rough ;) )

This is the original Meshroom draft Mesh:

I really like how NeuralRecon was able to reconstruct the part of the railing

[flolu]

@natowi I also ran into this problem and have a few questions regarding it:

(1) Over a year later, is it now possible to texture meshes with only the known camera positions?

(2) If it's still not possible, is this just a limitation of Meshroom? I mean is it possible by using the AliceVision framework directly?

(3) If it's also not possible with AliceVision, would it suffice to convert an existing high-resolution point cloud to depth maps and import them?

[natowi]

There is a project that supports ARKit import in Meshroom: https://github.com/SamMaoYS/arkit2meshroom (probably needs some updates to match the current release)

The latest release has some improvements for known poses, but I did not check out the details yet.