[question] How to modify (and reimport) the SFM data in another application?

[cnlohr]

I am currently considering writing a C VR application (as I have a lot of experience writing VR tools in C) for manual manipulation of point cloud data sets, but I am finding it very challenging to work with the Alembic format in a way that Meshroom tolerates. Importing/exporting to/from blender doesn't even work, with Meshroom. I get the following error message when trying to continue working in Meshroom:

Program called with the following parameters:
 * evCorrection = 0
 * input = "c:/Stuff/CabinAtNightCombine/MeshroomCache/StructureFromMotion/e6f328f692f893a86396e7341081a252fd751180/sfm.abc"
 * output = "c:/Stuff/CabinAtNightCombine/MeshroomCache/PrepareDenseScene/8ab071fd41723a9be9b7ef2f27c5401559dc74e6"
 * outputFileType = "exr"
 * rangeSize = 40
 * rangeStart = 40
 * saveMatricesTxtFiles = 0
 * saveMetadata = 1
 * verboseLevel = "info"

Exporting Scene Undistorted Images
0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
[02:37:17.272880][info] Median Camera Exposure: 5.61365e-36, Median EV: 117.1

I've also tried selecting .ply for the "Inter File Extension" in the "Structure From Motion" node, however, it seems that that only works on the files within the node, and it does not appear to be possible to finish the process without overwriting any modified ply's.

Or potentially some way to convert SFM data from PLYs back into ABCs as a node?

It would be incredible if there was another node for "finishing" structure from motion, since .ply's are much easier to work with than .abc's (even if not in Blender). Any hints on any way to modify the point cloud data in an external application?

Version:

• Windows 10, Meshroom 2021.1.0
• Bundled Python

[natowi]

convert SFM data from PLYs back into ABCs as a node

You can try the ConvertSfMFormat node.

Here you can find Blender export settings: https://github.com/alicevision/meshroom/wiki/Texturing-after-external-re-topology and this plugin may also be useful https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer

If you are looking for a way to add known poses, the creators of scAnt (an open-source platform for the creation of 3D models of arthropods(and other small objects)) wrote their own script to create a sfm file with estimated poses. It can be used as reference: https://github.com/evo-biomech/scAnt/blob/master/scripts/estimate_camera_positions.py

There is also https://github.com/alicevision/meshroom/wiki/Using-known-camera-positions but it needs to be updated. But there are links to other issues that may be helpful.

[cnlohr]

(1) I tried to use the Convert SFM node as such (with the left conversion to PLY and right to ABC) however, when given a PLY input, it returns the following error:

[16:59:38.381496][error] Unknown input SfM data format: '.ply'
[16:59:38.381997][error] The input SfMData file 'c:/Stuff/CabinAtNightCombine/MeshroomCache/ConvertSfMFormat/29640aec7adc9e93ab0f2efa73c7413017d05d1d/sfm.ply' cannot be read

I am finding that Meshroom is incredibly accurate and effective at estimating camera poses based on the existing pipelines, however, it needs human input for situations where it's impossible to see features on a surface. So I have no need for doing anything with camera positions. I just want to modify the point cloud.

[cnlohr]

It seems if I go through .sfm format files it is able to convert forward and back, which is excellent.

I realize now this may not work the way I expect. It seems that in order to have extra points for the rest of the system to use, you must provide a providence for that point, i.e. provide observations for the coordinate and color. Is that the case? Is that correct?

Do I need to provide which cameras are providing the point?

[cnlohr]

Wow. What is going on? If I use a .sfm and don't convert it back to a .abc basically every step of the pipeline takes 5x longer!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.