Meshroom icon indicating copy to clipboard operation
Meshroom copied to clipboard

Published 20 hours ago • verified publisher icon alicevision

Different performance between Meshrooms Windows and Linux

Open zell180 opened this issue 1 year ago • 18 comments

Hello all, we are experiencing very different behavior regarding the StructureFromMotion step in different environments.

We use the same dataset in the attached example. On Windows we use Meshroom 2023.1.0 and Windows 11 with Intel i9 12900KF as test environment

In production environment we use Ubuntu 20.04.04 LTS with 4x Intel Xeon Gold 5218

Operations in general are slightly slower on Xeon as per attached schematic but why does SFM take 10 times longer in production?

NODE XEON GOLD 5218 INTEL i9 12900KF DELTA
CameraInit 0,13 0,07 0,05
FeatureExtraction 8,77 7,34 1,43
ImageMatching 0,08 0,06 0,02
FeatureMatching 0,51 0,49 0,02
StructureFromMotion 76,39 7,13 69,26
SfMAlignment 0,10 0,09 0,01
PrepareDenseScene 5,17 4,84 0,33
DepthMap 17,97 11,69 6,28
DepthMapFilter 8,85 7,63 1,22
Meshing 31,81 25,23 6,58
MeshFiltering 3,50 2,99 0,51
Texturing 27,97 15,86 12,10

I attach the logs of the two environments. Is there any way to optimize the operation?

Thank you

LOG SVIL `[2023-07-24 11:07:08.363085] [0x000010a8] [trace] Embedded OCIO configuration file: 'C:\Users\user\Desktop\Meshroom-2023.1.0\aliceVision/share/aliceVision/config.ocio' found. Program called with the following parameters:

  • computeStructureColor = 1
  • describerTypes = "sift_float"
  • extraInfoFolder = "C:/Users/user/MeshroomCache/StructureFromMotion/2c3dd681fcd589009e23623dc173c06f1fe7483d"
  • featuresFolders = = [C:/Users/user/MeshroomCache/FeatureExtraction/4c1d00b2a3aacf9feaa409e8341566eac773266f]
  • filterTrackForks = 0
  • initialPairA = ""
  • initialPairB = ""
  • input = "C:/Users/user/MeshroomCache/CameraInit/feac85b8b45dcd0c20cbbce224d936e03ccf650c/cameraInit.sfm"
  • interFileExtension = ".abc"
  • localBAGraphDistance = 1
  • localizerEstimator = Unknown Type "enum aliceVision::robustEstimation::ERobustEstimator"
  • localizerEstimatorError = 0
  • localizerEstimatorMaxIterations = 4096
  • lockAllIntrinsics = 0
  • lockScenePreviouslyReconstructed = 0
  • matchesFolders = = [C:/Users/user/MeshroomCache/FeatureMatching/42c6c406b799b5f3318b78d46eb8b1b649dcbe1d]
  • maxAngleInitialPair = 40
  • maxCoresAvailable = Unknown Type "unsigned int" (default)
  • maxMemoryAvailable = 18446744073709551615 (default)
  • maxNumberOfMatches = 0
  • maxReprojectionError = 4
  • minAngleForLandmark = 2
  • minAngleForTriangulation = 3
  • minAngleInitialPair = 5
  • minInputTrackLength = 2
  • minNbCamerasToRefinePrincipalPoint = 3
  • minNumberOfMatches = 0
  • minNumberOfObservationsForTriangulation = 2
  • observationConstraint = Unknown Type "enum aliceVision::sfm::EFeatureConstraint"
  • output = "C:/Users/user/MeshroomCache/StructureFromMotion/2c3dd681fcd589009e23623dc173c06f1fe7483d/sfm.abc"
  • outputViewsAndPoses = "C:/Users/user/MeshroomCache/StructureFromMotion/2c3dd681fcd589009e23623dc173c06f1fe7483d/cameras.sfm"
  • randomSeed = 5489 (default)
  • rigMinNbCamerasForCalibration = 20
  • useLocalBA = 1
  • useOnlyMatchesFromInputFolder = 0
  • useRigConstraint = 1
  • verboseLevel = "info"

Hardware : Detected core count : 8 OpenMP will use 8 cores Detected available memory : 52967 Mo

[11:07:08.367085][warning] The number of intrinsics is incoherent: [11:07:08.367085][warning] 2 intrinsics declared and 1 intrinsics used. Loading features 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[11:07:09.436085][info] Fuse matches into tracks: - # tracks: 23152 - # images in tracks: 6 [11:07:09.436085][info] TrackLength, Occurrence [11:07:09.436085][info] 2 15415 [11:07:09.436085][info] 3 4354 [11:07:09.436085][info] 4 1862 [11:07:09.436085][info] 5 935 [11:07:09.437086][info] 6 586 Automatic selection of an initial pair: 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[11:07:10.081594][info] Initial pair is: - [A] view id: 286536366, filepath: C:/Users/user/Desktop/test/aaa.tiff - [B] view id: 316091842, filepath: C:/Users/user/Desktop/test/fff.tiff [11:07:10.083593][info] 1416 matches in the image pair for the initial pose estimation. [11:07:10.375592][info] Bundle adjustment start. [11:07:10.376592][info] Start bundle adjustment iteration: 0 [11:07:10.402592][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 0.0192927 s - poses: - # refined: 2 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 1310 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 0 - # constant: 1 - # ignored: 0 - # residual blocks: 2620 - # successful iterations: 7 - # unsuccessful iterations: 0 - initial RMSE: 0.363252 - final RMSE: 0.25659 [11:07:10.403595][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [11:07:10.403595][info] Bundle adjustment iteration: 0 took 27 msec. [11:07:10.403595][info] Bundle adjustment with 1 iterations took 27 msec. [11:07:10.403595][info] Initial pair is: 286536366, 316091842 [11:07:10.404594][info] Begin Incremental Reconstruction: - mode: SfM augmentation - # images in input: 6 - # images in resection: 4 - # landmarks in input: 655 - # cameras already calibrated: 2 [11:07:10.404594][info] Incremental Reconstruction start iteration 0: - # number of resection groups: 0 - # number of poses: 2 - # number of landmarks: 655 - # remaining images: 4 [11:07:10.404594][info] Update Reconstruction: - resection id: 0 - # images in the resection group: 1 - # images remaining: 4 [11:07:10.404594][info] [3/6] Robust Resection of view: 1094747651 [11:07:10.468592][info] Robust Resection information: - resection status: true - threshold (error max): 34.1947 - # points used for resection: 516 - # points validated by robust resection: 498 [11:07:10.494592][info] Bundle adjustment start. [11:07:10.495593][info] Start bundle adjustment iteration: 0 [11:07:10.796598][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 0.29616 s - poses: - # refined: 3 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 3504 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 7008 - # successful iterations: 49 - # unsuccessful iterations: 2 - initial RMSE: 0.530359 - final RMSE: 0.212086 [11:07:10.797599][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [11:07:10.797599][info] Bundle adjustment iteration: 0 took 302 msec. [11:07:10.797599][info] Bundle adjustment with 1 iterations took 302 msec. [11:07:10.805593][info] Update Reconstruction: - resection id: 1 - # images in the resection group: 1 - # images remaining: 3 [11:07:10.806593][info] [4/6] Robust Resection of view: 883902540 [11:07:10.932593][info] Robust Resection information: - resection status: true - threshold (error max): 12.2912 - # points used for resection: 1076 - # points validated by robust resection: 1029 [11:07:10.989593][info] Bundle adjustment start. [11:07:10.990593][info] Start bundle adjustment iteration: 0 [11:07:11.809593][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 0.801969 s - poses: - # refined: 4 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 11985 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 23970 - # successful iterations: 51 - # unsuccessful iterations: 0 - initial RMSE: 0.412076 - final RMSE: 0.239208 [11:07:11.811593][info] Remove outliers: - # outliers residual error: 1 - # outliers angular error: 0 [11:07:11.811593][info] Bundle adjustment iteration: 0 took 821 msec. [11:07:11.811593][info] Bundle adjustment with 1 iterations took 822 msec. [11:07:11.815593][info] Update Reconstruction: - resection id: 2 - # images in the resection group: 1 - # images remaining: 2 [11:07:11.816593][info] [5/6] Robust Resection of view: 1998982598 [11:07:12.122659][info] Robust Resection information: - resection status: true - threshold (error max): 11.9954 - # points used for resection: 2424 - # points validated by robust resection: 2306 [11:07:12.163592][info] Bundle adjustment start. [11:07:12.165592][info] Start bundle adjustment iteration: 0 [11:07:13.867594][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 1.66874 s - poses: - # refined: 5 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 23135 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 46270 - # successful iterations: 50 - # unsuccessful iterations: 1 - initial RMSE: 0.3449 - final RMSE: 0.192738 [11:07:13.870593][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [11:07:13.871594][info] Bundle adjustment iteration: 0 took 1706 msec. [11:07:13.871594][info] Bundle adjustment with 1 iterations took 1707 msec. [11:07:13.877592][info] Update Reconstruction: - resection id: 3 - # images in the resection group: 1 - # images remaining: 1 [11:07:13.880592][info] [6/6] Robust Resection of view: 1081437535 [11:07:14.350659][info] Robust Resection information: - resection status: true - threshold (error max): 8.38322 - # points used for resection: 3581 - # points validated by robust resection: 3361 [11:07:14.513655][info] Bundle adjustment start. [11:07:14.516655][info] Start bundle adjustment iteration: 0 [11:07:16.830598][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 2.26485 s - poses: - # refined: 6 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 34142 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 68284 - # successful iterations: 51 - # unsuccessful iterations: 0 - initial RMSE: 0.314972 - final RMSE: 0.157576 [11:07:16.835594][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [11:07:16.836594][info] Bundle adjustment iteration: 0 took 2319 msec. [11:07:16.836594][info] Bundle adjustment with 1 iterations took 2322 msec. [11:07:16.845592][info] Incremental Reconstruction start iteration 1: - # number of resection groups: 4 - # number of poses: 6 - # number of landmarks: 11812 - # remaining images: 0 [11:07:16.845592][info] Incremental Reconstruction completed with 2 iterations: - # number of resection groups: 4 - # number of poses: 6 - # number of landmarks: 11812 - # remaining images: 0 [11:07:16.848594][info] Structure from Motion statistics: - # input images: 6 - # cameras calibrated: 6 - # poses: 6 - # landmarks: 11812 - elapsed time: 6.441 - residual RMSE: 0.707855 [11:07:16.849593][info] - # sift_float: 11812 [11:07:16.853593][info] Histogram of residuals:

0 | 1.5e+04 0.5 | 1.1e+04 1 | 5.1e+03 1.5 | 2e+03 2 | 8.1e+02 2.5 | 4.2e+02 3 | 1.8e+02 3.5 | 97 4 | 38 4.5 | 31 5 | 19 5.5 | 9 6 | 8 6.5 | 6 7 | 6 7.5 | 0 8 | 1 8.5 | 0 9 [11:07:16.853593][info] # landmarks: 11812 [11:07:16.853593][info] # overall observations: 34142 [11:07:16.853593][info] Landmarks observations length min: 2, mean: 2.89045, median: 2, max: 6 [11:07:16.853593][info] Histogram of observations length:

2 | 6015 3 | 2933 4 | 1510 5 | 851 6 | 503 7 [11:07:16.854592][info] Landmarks per view min: 0, mean: 0, median: 0, max: 0 [11:07:16.854592][info] Histogram of nb landmarks per view:

0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 1

Compute scene structure color 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[11:07:17.488593][info] Structure from motion took (s): 8.140000 [11:07:17.488593][info] Generating HTML report... [11:07:17.498593][info] Export SfMData to disk: C:/Users/user/MeshroomCache/StructureFromMotion/2c3dd681fcd589009e23623dc173c06f1fe7483d/sfm.abc [11:07:17.510593][info] Structure from Motion results: - # input images: 6 - # cameras calibrated: 6 - # poses: 6 - # landmarks: 11812 `

LOG PROD `[2023-07-19 08:40:51.463769] [0x00007fcd87b15000] [trace] Embedded OCIO configuration file: '/Meshroom-2023.1.0-av3.0.0-centos7-cuda11.3.1/aliceVision/share/aliceVision/config.ocio' found. Program called with the following parameters:

  • computeStructureColor = 1
  • describerTypes = "sift_float"
  • extraInfoFolder = "/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/StructureFromMotion/7f9c2cf20d68b31efc56e47937f5465cec9bdd77"
  • featuresFolders = = [/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/FeatureExtraction/7dd58d736ce1adf534fda32ea7d7eba078b2750b]
  • filterTrackForks = 0
  • initialPairA = ""
  • initialPairB = ""
  • input = "/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/CameraInit/2ad315f9152a75f6c5401b8d7ae3e8604b4a97f3/cameraInit.sfm"
  • interFileExtension = ".abc"
  • localBAGraphDistance = 1
  • localizerEstimator = Unknown Type "N11aliceVision16robustEstimation16ERobustEstimatorE"
  • localizerEstimatorError = 0
  • localizerEstimatorMaxIterations = 4096
  • lockAllIntrinsics = 0
  • lockScenePreviouslyReconstructed = 0
  • matchesFolders = = [/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/FeatureMatching/ee2cc8d742241335253bf5b371de8f050693fffb]
  • maxAngleInitialPair = 40
  • maxCoresAvailable = Unknown Type "j"
  • maxMemoryAvailable = 9223372036854771712
  • maxNumberOfMatches = 0
  • maxReprojectionError = 4
  • minAngleForLandmark = 2
  • minAngleForTriangulation = 3
  • minAngleInitialPair = 5
  • minInputTrackLength = 2
  • minNbCamerasToRefinePrincipalPoint = 3
  • minNumberOfMatches = 0
  • minNumberOfObservationsForTriangulation = 2
  • observationConstraint = Unknown Type "N11aliceVision3sfm18EFeatureConstraintE"
  • output = "/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/StructureFromMotion/7f9c2cf20d68b31efc56e47937f5465cec9bdd77/sfm.abc"
  • outputViewsAndPoses = "/test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/StructureFromMotion/7f9c2cf20d68b31efc56e47937f5465cec9bdd77/cameras.sfm"
  • randomSeed = 5489 (default)
  • rigMinNbCamerasForCalibration = 20
  • useLocalBA = 1
  • useOnlyMatchesFromInputFolder = 0
  • useRigConstraint = 1
  • verboseLevel = "info"

Hardware : Detected core count : 128 User upper limit on core count : 128 OpenMP will use 128 cores Detected available memory : 183428 Mo User upper limit on memory available : 8796093022207 Mo

Loading features 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[08:40:51.945808][info] Fuse matches into tracks: - # tracks: 23313 - # images in tracks: 6 [08:40:51.946345][info] TrackLength, Occurrence [08:40:51.946402][info] 2 15488 [08:40:51.946415][info] 3 4413 [08:40:51.946423][info] 4 1848 [08:40:51.946430][info] 5 952 [08:40:51.946437][info] 6 612 Automatic selection of an initial pair: 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[08:40:52.500611][info] Initial pair is: - [A] view id: 376464373, filepath: /test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/tiff/ddd.tiff - [B] view id: 2068822876, filepath: /test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/tiff/aaa.tiff [08:40:52.504749][info] 2481 matches in the image pair for the initial pose estimation. [08:40:53.149681][info] Bundle adjustment start. [08:40:53.151093][info] Start bundle adjustment iteration: 0 block_sparse_matrix.cc:92 Allocating values array with 529632 bytes. detect_structure.cc:114 Schur complement static structure <2,3,6>. detect_structure.cc:114 Schur complement static structure <2,3,6>. [08:40:54.543160][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 1.36934 s - poses: - # refined: 2 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 3678 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 0 - # constant: 1 - # ignored: 0 - # residual blocks: 7356 - # successful iterations: 5 - # unsuccessful iterations: 0 - initial RMSE: 0.286364 - final RMSE: 0.265171 [08:40:54.552676][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [08:40:54.552877][info] Bundle adjustment iteration: 0 took 1401 msec. [08:40:54.552891][info] Bundle adjustment with 1 iterations took 1403 msec. [08:40:54.557233][info] Initial pair is: 376464373, 2068822876 [08:40:54.559000][info] Begin Incremental Reconstruction: - mode: SfM augmentation - # images in input: 6 - # images in resection: 4 - # landmarks in input: 1839 - # cameras already calibrated: 2 [08:40:54.559026][info] Incremental Reconstruction start iteration 0: - # number of resection groups: 0 - # number of poses: 2 - # number of landmarks: 1839 - # remaining images: 4 [08:40:54.569957][info] Update Reconstruction: - resection id: 0 - # images in the resection group: 1 - # images remaining: 4 [08:40:54.583626][info] [3/6] Robust Resection of view: 1570894053 [08:40:54.803290][info] Robust Resection information: - resection status: true - threshold (error max): 19.3737 - # points used for resection: 1366 - # points validated by robust resection: 1341 [08:40:55.913850][info] Bundle adjustment start. [08:40:55.915273][info] Start bundle adjustment iteration: 0 block_sparse_matrix.cc:92 Allocating values array with 1596240 bytes. detect_structure.cc:114 Schur complement static structure <2,3,6>. detect_structure.cc:114 Schur complement static structure <2,3,6>. [08:41:33.537495][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 37.598 s - poses: - # refined: 3 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 6651 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 13302 - # successful iterations: 51 - # unsuccessful iterations: 0 - initial RMSE: 0.438155 - final RMSE: 0.097772 [08:41:33.544681][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [08:41:33.545205][info] Bundle adjustment iteration: 0 took 37629 msec. [08:41:33.545219][info] Bundle adjustment with 1 iterations took 37631 msec. [08:41:33.561082][info] Update Reconstruction: - resection id: 1 - # images in the resection group: 1 - # images remaining: 3 [08:41:33.573598][info] [4/6] Robust Resection of view: 1867449040 [08:41:33.844934][info] Robust Resection information: - resection status: true - threshold (error max): 3.62351 - # points used for resection: 1787 - # points validated by robust resection: 1691 [08:41:34.725754][info] Bundle adjustment start. [08:41:34.728752][info] Start bundle adjustment iteration: 0 block_sparse_matrix.cc:92 Allocating values array with 5071440 bytes. detect_structure.cc:114 Schur complement static structure <2,3,6>. detect_structure.cc:114 Schur complement static structure <2,3,6>. [08:41:52.075487][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 17.2932 s - poses: - # refined: 4 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 21131 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 42262 - # successful iterations: 28 - # unsuccessful iterations: 0 - initial RMSE: 0.201421 - final RMSE: 0.104666 [08:41:52.091352][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [08:41:52.092505][info] Bundle adjustment iteration: 0 took 17363 msec. [08:41:52.092522][info] Bundle adjustment with 1 iterations took 17366 msec. [08:41:52.112607][info] Update Reconstruction: - resection id: 2 - # images in the resection group: 1 - # images remaining: 2 [08:41:52.126313][info] [5/6] Robust Resection of view: 1923288372 [08:41:52.595461][info] Robust Resection information: - resection status: true - threshold (error max): 4.99661 - # points used for resection: 2969 - # points validated by robust resection: 2768 [08:41:53.207456][info] Bundle adjustment start. [08:41:53.212141][info] Start bundle adjustment iteration: 0 block_sparse_matrix.cc:92 Allocating values array with 7801920 bytes. detect_structure.cc:114 Schur complement static structure <2,3,6>. detect_structure.cc:114 Schur complement static structure <2,3,6>. [08:41:59.230085][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 5.92496 s - poses: - # refined: 5 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 32508 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 65016 - # successful iterations: 9 - # unsuccessful iterations: 0 - initial RMSE: 0.162355 - final RMSE: 0.110001 [08:41:59.247608][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [08:41:59.249100][info] Bundle adjustment iteration: 0 took 6036 msec. [08:41:59.249123][info] Bundle adjustment with 1 iterations took 6041 msec. [08:41:59.265331][info] Update Reconstruction: - resection id: 3 - # images in the resection group: 1 - # images remaining: 1 [08:41:59.275838][info] [6/6] Robust Resection of view: 2144614340 [08:41:59.888453][info] Robust Resection information: - resection status: true - threshold (error max): 5.12541 - # points used for resection: 3534 - # points validated by robust resection: 3285 [08:42:00.496622][info] Bundle adjustment start. [08:42:00.501668][info] Start bundle adjustment iteration: 0 block_sparse_matrix.cc:92 Allocating values array with 10151040 bytes. detect_structure.cc:114 Schur complement static structure <2,3,6>. detect_structure.cc:114 Schur complement static structure <2,3,6>. [08:42:07.304194][info] Bundle Adjustment Statistics: - local strategy enabled: no - adjustment duration: 6.68326 s - poses: - # refined: 6 - # constant: 0 - # ignored: 0 - landmarks: - # refined: 42296 - # constant: 0 - # ignored: 0 - intrinsics: - # refined: 1 - # constant: 0 - # ignored: 0 - # residual blocks: 84592 - # successful iterations: 10 - # unsuccessful iterations: 0 - initial RMSE: 0.151011 - final RMSE: 0.11542 [08:42:07.321970][info] Remove outliers: - # outliers residual error: 0 - # outliers angular error: 0 [08:42:07.323830][info] Bundle adjustment iteration: 0 took 6822 msec. [08:42:07.323849][info] Bundle adjustment with 1 iterations took 6827 msec. [08:42:07.336875][info] Incremental Reconstruction start iteration 1: - # number of resection groups: 4 - # number of poses: 6 - # number of landmarks: 15036 - # remaining images: 0 [08:42:07.336897][info] Incremental Reconstruction completed with 2 iterations: - # number of resection groups: 4 - # number of poses: 6 - # number of landmarks: 15036 - # remaining images: 0 [08:42:07.340945][info] Structure from Motion statistics: - # input images: 6 - # cameras calibrated: 6 - # poses: 6 - # landmarks: 15036 - elapsed time: 72.777 - residual RMSE: 0.581205 [08:42:07.341856][info] - # sift_float: 15036 [08:42:07.349106][info] Histogram of residuals:

0 | 2.4e+04 0.5 | 1.1e+04 1 | 4.2e+03 1.5 | 1.5e+03 2 | 6.2e+02 2.5 | 3e+02 3 | 1.5e+02 3.5 | 76 4 | 48 4.5 | 25 5 | 15 5.5 | 8 6 | 7 6.5 | 0 7 | 1 7.5 | 1 8 [08:42:07.349841][info] # landmarks: 15036 [08:42:07.349851][info] # overall observations: 42296 [08:42:07.349857][info] Landmarks observations length min: 2, mean: 2.81298, median: 2, max: 6 [08:42:07.349866][info] Histogram of observations length:

2 | 8178 3 | 3598 4 | 1747 5 | 920 6 | 593 7 [08:42:07.350189][info] Landmarks per view min: 0, mean: 0, median: 0, max: 0 [08:42:07.350202][info] Histogram of nb landmarks per view:

0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 0 | 0 1

Compute scene structure color 0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----|

[08:42:07.989449][info] Structure from motion took (s): 76.394000 [08:42:07.989558][info] Generating HTML report... [08:42:08.005434][info] Export SfMData to disk: /test/data/168975324090_Pioltello_LL_dispari_not_available/SD_OUT/ms_out/StructureFromMotion/7f9c2cf20d68b31efc56e47937f5465cec9bdd77/sfm.abc [08:42:08.015161][info] Structure from Motion results: - # input images: 6 - # cameras calibrated: 6 - # poses: 6 - # landmarks: 15036 `

zell180 avatar Jul 24 '23 09:07 zell180

No one have opinion about this behaviour?

zell180 avatar Aug 02 '23 13:08 zell180

Not sure why there is such a large time difference. Doing a compare between the logs I can see a variety of differences, such as the number of landmarks found at the end of the SFM step. I would have expected this to be the same given the same input data.

Is the feature detection and matching producing identical results on both systems?

I would try to eliminate any potential differences in the process. For the SFM step use the same initial image pair (on windows it used aaa.tiff and fff.tiff, while on linux it used ddd.tiff and aaa.tiff). Not sure if there is a way to limit the number of CPUs used, but if there is then try to use 8 for both.

msanta avatar Aug 06 '23 01:08 msanta

Not sure why there is such a large time difference. Doing a compare between the logs I can see a variety of differences, such as the number of landmarks found at the end of the SFM step. I would have expected this to be the same given the same input data.

Is the feature detection and matching producing identical results on both systems?

I would try to eliminate any potential differences in the process. For the SFM step use the same initial image pair (on windows it used aaa.tiff and fff.tiff, while on linux it used ddd.tiff and aaa.tiff). Not sure if there is a way to limit the number of CPUs used, but if there is then try to use 8 for both.

About saome initial pair i've try to set static pair and the result is the same about elapsed time. I was just looking for a way to limit the cpu, someone know how to do that?

zell180 avatar Aug 21 '23 15:08 zell180

I've done new test with i7-1165G7. Same photo and same settings on Windows. With normal and ultra respectively i got 9 sec and 12 sec for StructureFromMotion. I think number of cores should be the problem. How can i limit it on that node? @fabiencastan

zell180 avatar Aug 28 '23 13:08 zell180

I think there is a way to add this limitation, but I don´t know details. Related PRs: https://github.com/alicevision/AliceVision/pull/1304 https://github.com/alicevision/Meshroom/pull/1836

natowi avatar Aug 28 '23 15:08 natowi

Ok i'm 99.9% sure that the problem are the large number of core. Just try with i5-8250U with 16GB RAM on Ubuntu 22.04 and the task took only 13.4seconds. Please someone can help to reduce core numbers or think a solution?

zell180 avatar Aug 28 '23 16:08 zell180

Maybe https://manpages.ubuntu.com/manpages/trusty/man1/cpulimit.1.html

The  -c flag sets the number of CPU cores the program thinks are available. Usually
              this is detected for us, but can be over-ridden.

natowi avatar Aug 28 '23 16:08 natowi

i got: Unrecoginzed argument -c 8 for meshroom_batch

zell180 avatar Aug 29 '23 07:08 zell180

Just tested on several computers, 16/40/96/256 cores and they are all the same runtime around 1m and 20 sec for 46 images.

servantftechnicolor avatar Aug 29 '23 08:08 servantftechnicolor

Just tested on several computers, 16/40/96/256 cores and they are all the same runtime around 1m and 20 sec for 46 images.

i've only 6 image and the runtime gap is crazy. What should i check or fix?

zell180 avatar Aug 29 '23 08:08 zell180

so there is no way to reduce cpu number?

zell180 avatar Sep 04 '23 11:09 zell180

How did you run the command?

msanta avatar Sep 04 '23 12:09 msanta

The cpulimit command should be used something like this: cpulimit -c 8 -l 100 -- /path/to/meshroom_batch <options for meshroom_batch>. However meshroom_batch is going to start another process for each step and I don't think these will have the CPU limit applied.

Interestingly I had a look at the .status files from a project and there is a --maxCores flag specified when running the individual commands. Eg: "commandLine": "aliceVision_featureMatching --input \"...\" --featuresFolders \"...\" --imagePairsList \"...\" --describerTypes dspsift --photometricMatchingMethod ANN_L2 --geometricEstimator acransac --geometricFilterType fundamental_matrix --distanceRatio 0.8 --maxIteration 2048 --geometricError 0.0 --knownPosesGeometricErrorMax 5.0 --minRequired2DMotion -1.0 --maxMatches 0 --savePutativeMatches False --crossMatching False --guidedMatching False --matchFromKnownCameraPoses False --exportDebugFiles False --verboseLevel info --output \"..\" --rangeStart 0 --rangeSize 20 --maxMemory=9223372036854771712 --maxCores=16"

msanta avatar Sep 04 '23 13:09 msanta

i've successfully launch meshroom_batch with the instruction you kindly provided but as you supspected the limit is ignored in meshroom "subprocess". The steps in which we have bad performance is StructureFromMotion, and seems that is not possible to pass this parameter. Is possible to do some mod to enable them?

zell180 avatar Sep 05 '23 07:09 zell180

You can use cpulimit on a running process by specifying the process ID with the -p option: cpulimit -c 8 -l 100 -p 1234. That should let you see if reducing the CPU resources has an impact.

The -e and -P options might be worth trying out. cpulimit will wait for a process with the given name (-e) or path (-P) and then throttle it. For example I can run the command to target Meshroom and its waits until I have started the app.

$ cpulimit -c 1 -l 100 -e Meshroom
Warning: no target process found. Waiting for it...
Process 60909 detected

However I have had no luck when targetting the aliceVision_incrementalSfM program by name or path (but specifying the process ID works of course).

msanta avatar Sep 05 '23 10:09 msanta

i've used cpulimit -c 8 -l 100 -e aliceVision_incrementalSfM and i've got Warning: no target process found. Waiting for it... Process 1144209 detected

Bingo i think! But in aliceVision_incrementalSfM log i see that cores are 128. I think we need to find a way to pass --maxCores to aliceVision_incrementalSfM

[5/12] StructureFromMotion

  • commandLine: aliceVision_incrementalSfM --input "cache/CameraInit/fdda14dd88f48c2b8510f134908d15a1d3287342/cameraInit.sfm" --featuresFolders "/cache/FeatureExtraction/c60c4b21c4cfa315367fa9e5661bd08460ac9094" --matchesFolders "/cache/FeatureMatching/679dd19955f1a0e47bf4c90888b20eb5aea72e59" --describerTypes sift_float --localizerEstimator acransac --observationConstraint Scale --localizerEstimatorMaxIterations 4096 --localizerEstimatorError 0.0 --lockScenePreviouslyReconstructed False --useLocalBA True --localBAGraphDistance 1 --maxNumberOfMatches 0 --minNumberOfMatches 0 --minInputTrackLength 2 --minNumberOfObservationsForTriangulation 2 --minAngleForTriangulation 3.0 --minAngleForLandmark 2.0 --maxReprojectionError 4.0 --minAngleInitialPair 5.0 --maxAngleInitialPair 40.0 --useOnlyMatchesFromInputFolder False --useRigConstraint True --rigMinNbCamerasForCalibration 20 --lockAllIntrinsics False --minNbCamerasToRefinePrincipalPoint 3 --filterTrackForks False --computeStructureColor True --initialPairA "aaa.tiff" --initialPairB "ddd.tiff" --interFileExtension .abc --verboseLevel info --output "/cache/StructureFromMotion/0f2dd1f6f46707ec4815a6b4b8b6cd2c6b036bcf/sfm.abc" --outputViewsAndPoses "/cache/StructureFromMotion/0f2dd1f6f46707ec4815a6b4b8b6cd2c6b036bcf/cameras.sfm" --extraInfoFolder " //cache/StructureFromMotion/0f2dd1f6f46707ec4815a6b4b8b6cd2c6b036bcf" --maxMemory=9223372036854771712 --maxCores=128

zell180 avatar Sep 06 '23 09:09 zell180

--maxCores is set here for all the command lines https://github.com/alicevision/Meshroom/blob/ea26d89844456883fc995a76115dd701cc51ec10/meshroom/core/desc.py#L680 it reads the value from meshroom's cgroup Does aliceVision_incrementalSfM really access to the all cores?

FlachyJoe avatar Sep 06 '23 12:09 FlachyJoe

if i read aliceVision_incrementalSfM help i see that param is not --maxCores but --maxMemoryAvailable and --maxCoresAvailable. Should be this the problem?

zell180 avatar Sep 06 '23 12:09 zell180