TheiaSfM
TheiaSfM copied to clipboard
sweeneychris
Bundle adjustment failed!
I get some errors about BA when using incremental SFM, log as follow:
I1009 21:16:01.156069 2348131200 incremental_reconstruction_estimator.cc:482] Running full bundle adjustment on the entire reconstruction.
I1009 21:16:01.161542 2348131200 incremental_reconstruction_estimator.cc:510] Selected 538 tracks to optimize.
I1009 21:16:01.164420 2348131200 bundle_adjuster.cc:206]
Solver Summary (v 1.14.0-eigen-(3.3.5)-lapack-suitesparse-(5.3.0)-cxsparse-(3.1.9)-eigensparse-no_openmp-no_tbb)
Original Reduced
Parameter blocks 543 543
Parameters 2183 2183
Effective parameters 2179 2179
Residual blocks 1242 1242
Residuals 2484 2484
Minimizer TRUST_REGION
Sparse linear algebra library SUITE_SPARSE
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_SCHUR SPARSE_NORMAL_CHOLESKY
Threads 1 1
Linear solver ordering 538,1,5 538,1,5
Cost:
Initial -1.000000e+00
Minimizer iterations -2
Successful steps -1
Unsuccessful steps -1
Time (in seconds):
Preprocessor 0.000115
Residual only evaluation -1.000000 (-1)
Jacobian & residual evaluation -1.000000 (-1)
Linear solver -1.000000 (-1)
Minimizer -1.000000
Postprocessor 0.000039
Total 0.000154
Termination: FAILURE (The program has 543 parameter blocks, but the parameter block ordering has 544 parameter blocks.)
I1009 21:16:01.178530 2348131200 set_outlier_tracks_to_unestimated.cc:128] 525 points were removed because of bad reprojection errors. 0 points were removed because they had insufficient viewing angles and were poorly constrained.
I1009 21:16:01.178570 2348131200 incremental_reconstruction_estimator.cc:607] 525 outlier points were removed.
W1009 21:16:01.189359 2348131200 incremental_reconstruction_estimator.cc:276] Bundle adjustment failed!
Thanks
what are the flags you are running this with? looks like you are optimizing focal length with shared intrinsics maybe?
I have a dataset that will fail to bundle adjust using incremental SFM if shared calibration enabled.
Once I disabled shared calibration, the incremental SFM bundle adjusts worked and I obtained a good reconstruction. This is with focal length and radial distortion optimization enabled.
I can only say that in my case the camera is... submerged (under water) and thus various distortion effects may be at play.
My incremental reconstruction test pass, The following tests FAILED: 13 - find_polynomial_roots_jenkins_traub_test (Failed) 16 - normalized_graph_cut_test (Failed) 53 - compute_triplet_baseline_ratios_test (SEGFAULT) 54 - least_unsquared_deviation_position_estimator_test (Failed) 55 - linear_position_estimator_test (SEGFAULT) 58 - pairwise_rotation_error_test (SEGFAULT) 78 - sim_transform_partial_rotation_test (Failed) 91 - orientations_from_maximum_spanning_tree_test (SEGFAULT) Errors while running CTest
Interesting. Some of those errors I have also seen but could never actually repeat them to debug.
I will look into the shared intrinsic error this week. Looks like a parameter is being added but not used.
If either of you have a sample dataset and flags to reproduce this please send them my way as that would be extremely helpful!
I'll have to give permission to share the dataset with you. That being said I could probably share a few of the images with you and talk about what the images are and how they were collected and what the environment is. I have a few thoughts as to why bundle adjust is filling with shared calibration. But I guess I'll ask this first:
I recently read that you can have about a 2-degree difference in refraction angle with 10° increase temperature underwater. In regard to the inference angle converted to a refraction angle of light moving through Air versus water.
How viable is it that shared camera intrinsics would fail due to variance in the distortion in the underwater environment? There is a lot of equipment involved and I can see heat being generated in cold underwater temperatures causing pockets of warmer water throughout the scene
On Mon, Oct 15, 2018, 11:03 AM Chris Sweeney [email protected] wrote:
Interesting. Some of those errors I have also seen but could never actually repeat them to debug.
I will look into the shared intrinsic error this week. Looks like a parameter is being added but not used.
If either of you have a sample dataset and flags to reproduce this please send them my way as that would be extremely helpful!
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-429890247, or mute the thread https://github.com/notifications/unsubscribe-auth/AFGI5kcfGAHz8iuVw23jA82jvydwXxI7ks5ulKPQgaJpZM4XTTGl .
"The program has 543 parameter blocks, but the parameter block ordering has 544 parameter blocks."
This error message from Ceres makes me believe that somehow the Bundle adjustment problem is just not being assembled correctly. 1 extra pesky parameter block is something sneaking into the optimizer so that's what I need to track down. I don't think that error is related to the input sfm quality, but I might be wrong.
On Mon, Oct 15, 2018, 4:31 PM Charles Ott [email protected] wrote:
I'll have to give permission to share the dataset with you. That being said I could probably share a few of the images with you and talk about what the images are and how they were collected and what the environment is. I have a few thoughts as to why bundle adjust is filling with shared calibration. But I guess I'll ask this first:
I recently read that you can have about a 2-degree difference in refraction angle with 10° increase temperature underwater. In regard to the inference angle converted to a refraction angle of light moving through Air versus water.
How viable is it that shared camera intrinsics would fail due to variance in the distortion in the underwater environment? There is a lot of equipment involved and I can see heat being generated in cold underwater temperatures causing pockets of warmer water throughout the scene
On Mon, Oct 15, 2018, 11:03 AM Chris Sweeney [email protected] wrote:
Interesting. Some of those errors I have also seen but could never actually repeat them to debug.
I will look into the shared intrinsic error this week. Looks like a parameter is being added but not used.
If either of you have a sample dataset and flags to reproduce this please send them my way as that would be extremely helpful!
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-429890247 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AFGI5kcfGAHz8iuVw23jA82jvydwXxI7ks5ulKPQgaJpZM4XTTGl
.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-430049455, or mute the thread https://github.com/notifications/unsubscribe-auth/AAwytusQiDpMQ0UGroqtGkE3o_BhZ3Bkks5ulRrtgaJpZM4XTTGl .
Got it. I will try to repro the error tomorrow and see what I can capture in the debugger.
On Mon, Oct 15, 2018, 8:10 PM Chris Sweeney [email protected] wrote:
"The program has 543 parameter blocks, but the parameter block ordering has 544 parameter blocks."
This error message from Ceres makes me believe that somehow the Bundle adjustment problem is just not being assembled correctly. 1 extra pesky parameter block is something sneaking into the optimizer so that's what I need to track down. I don't think that error is related to the input sfm quality, but I might be wrong.
On Mon, Oct 15, 2018, 4:31 PM Charles Ott [email protected] wrote:
I'll have to give permission to share the dataset with you. That being said I could probably share a few of the images with you and talk about what the images are and how they were collected and what the environment is. I have a few thoughts as to why bundle adjust is filling with shared calibration. But I guess I'll ask this first:
I recently read that you can have about a 2-degree difference in refraction angle with 10° increase temperature underwater. In regard to the inference angle converted to a refraction angle of light moving through Air versus water.
How viable is it that shared camera intrinsics would fail due to variance in the distortion in the underwater environment? There is a lot of equipment involved and I can see heat being generated in cold underwater temperatures causing pockets of warmer water throughout the scene
On Mon, Oct 15, 2018, 11:03 AM Chris Sweeney [email protected] wrote:
Interesting. Some of those errors I have also seen but could never actually repeat them to debug.
I will look into the shared intrinsic error this week. Looks like a parameter is being added but not used.
If either of you have a sample dataset and flags to reproduce this please send them my way as that would be extremely helpful!
— You are receiving this because you commented. Reply to this email directly, view it on GitHub <
https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-429890247
, or mute the thread <
https://github.com/notifications/unsubscribe-auth/AFGI5kcfGAHz8iuVw23jA82jvydwXxI7ks5ulKPQgaJpZM4XTTGl
.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub < https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-430049455 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAwytusQiDpMQ0UGroqtGkE3o_BhZ3Bkks5ulRrtgaJpZM4XTTGl
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sweeneychris/TheiaSfM/issues/211#issuecomment-430056496, or mute the thread https://github.com/notifications/unsubscribe-auth/AFGI5kqXATKqDQC-YAZm1deYLi8I5JLNks5ulSPwgaJpZM4XTTGl .
@charlescva anything turn up here? I haven't traced it down from my end yet
Has this problem been solved ? I think I'm running into the same issue where Ceres keeps adding 1 extra block. In my case it happens during Incremental BA when I add principle point to the parameters to be optimized, on datasets that have very high focal lengths. If I omit principle point, it solves for all the images in the collection. When I use Global reconstruction + principle point optimization, Theia solves, but only for about half the camera poses.
Same issue here. BA fails after Ceres adds one extra parameter block during incremental SFM using shared calibration.
I20201005 13:26:59.011030 8948 incremental_reconstruction_estimator.cc:482] Running full bundle adjustment on the entire reconstruction.
I20201005 13:26:59.021042 8948 incremental_reconstruction_estimator.cc:510] Selected 635 tracks to optimize.
I20201005 13:26:59.026028 8948 bundle_adjuster.cc:206]
Solver Summary (v 2.0.0-eigen-(3.3.5)-lapack-suitesparse-(5.1.0)-no_openmp)
Original Reduced
Parameter blocks 639 639
Parameters 2568 2568
Effective parameters 2567 2567
Residual blocks 1270 1270
Residuals 2540 2540
Minimizer TRUST_REGION
Sparse linear algebra library SUITE_SPARSE
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_SCHUR SPARSE_NORMAL_CHOLESKY
Threads 1 1
Linear solver ordering 635,1,4 635,1,4
Cost:
Initial -1.000000e+00
Minimizer iterations -2
Successful steps -1
Unsuccessful steps -1
Time (in seconds):
Preprocessor 0.000114
Residual only evaluation 1.000000 (-1)
Jacobian & residual evaluation -1.000000 (-1)
Linear solver -1.000000 (-1)
Minimizer -1.000000
Postprocessor 0.000022
Total 0.000136
Termination: FAILURE (The program has 639 parameter blocks, but the parameter block ordering has 640 parameter blocks.)
I20201005 13:26:59.036211 8948 set_outlier_tracks_to_unestimated.cc:128] 225 points were removed because of bad reprojection errors. 0 points were removed because they had insufficient viewing angles and were poorly constrained.
I20201005 13:26:59.037204 8948 incremental_reconstruction_estimator.cc:607] 225 outlier points were removed.
W20201005 13:26:59.039225 8948 incremental_reconstruction_estimator.cc:276] Bundle adjustment failed!
F20201005 13:26:59.040030 8948 main.cpp:488] Check failed: reconstruction_builder.BuildReconstruction(&reconstructions) Could not create a reconstruction.
*** Check failure stack trace: ***
@ 00007FFBDA746265 public: void __cdecl google::LogMessage::Flush(void) __ptr64
@ 00007FFBDA744ED2 public: __cdecl google::LogMessageFatal::~LogMessageFatal(void) __ptr64
@ 00007FF6D254A6F2 main
@ 00007FF6D25A63B8 __scrt_common_main_seh
@ 00007FFC1A3B7BD4 BaseThreadInitThunk
@ 00007FFC1AF6CE51 RtlUserThreadStart
I’m using the tool calibrate_camera_intrinsics.cc to calibrate a camera having a short focal length (9 mm) with 21 images.
In my case I use the Pinhole Radial Tangential camera model and try to optimize simultaneously the focal length + principal point + skew + radial & tangential parameters. However, I couldn’t find out how to reproduce this error as it happens to occur sporadically on a same dataset with identical options. Reducing the number of parameters to estimate seems to reduce de probability to run into this error but with no guarantee of success. I haven’t noticed this issue using global SFM yet.
I can provide you with the dataset and the flags I used to get this error if this is still helpful.
