Problems with Large Focal Lengths

[ambarak]

First of all, thanks for this cool API, it's an awesome sandbox.

The problem I would like help with, I have a collection of 47 images taken with a camera of focal length 800mm Theia manages to get through two_view_geometric_verification, and bundle adjust the two_view_info with reasonable results, but then starts to give me the following messages: I0425 12:49:04.235623 31351 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits.

and it goes on like that for awhile before giving:

I0425 12:49:04.320199 31351 localize_view_to_reconstruction.cc:240] Failed to localize view id 0 with only 0 out of 31 features as inliers. I0425 12:49:04.321336 31351 reconstruction_builder.cc:384] Reconstruction estimation statistics: Num estimated views = 2 Num input views = 47 Num estimated tracks = 295 Num input tracks = 12067 Pose estimation time = 0.003196 Triangulation time = 0 Bundle Adjustment time = 0 Total time = 0.258344

Incremental Reconstruction Estimator timings: Time to find an initial seed for the reconstruction: 0.168333 I0425 12:49:04.340247 31351 sparse_reconstruction.cc:542] Num views: 2 Num 3D points: 295 I0425 12:49:04.340484 31351 print_reconstruction_statistics.h:83] Num observations: 590 Num reprojections behind camera: 0 Mean reprojection error = 0.742716 Median reprojection_error = 0.492501 I0425 12:49:04.340512 31351 print_reconstruction_statistics.h:112] Mean track length: 2 I0425 12:49:04.340518 31351 print_reconstruction_statistics.h:119] Median track length: 2 I0425 12:49:04.340534 31351 print_reconstruction_statistics.h:123] Track length histogram = [2 - 3) = 295 [3 - 4) = 0 [4 - 5) = 0 [5 - 6) = 0 [6 - 7) = 0 [7 - 8) = 0 [8 - 9) = 0 [9 - 10) = 0 [10 - 15) = 0 [15 - 20) = 0 [20 - 25) = 0 [25 - 50) = 0

I understand the problems of dealing with narrow field of view. What I'd like help with is advise on where and how to modify Theia to try and solve this. knowing that I have very accurate gps and orientation of the cameras, plus focal length.

Thanks! in advance.

[sweeneychris]

Did you try using incremetal Sfm?

[ambarak]

Yes I have,

What looks interesting is, pair-wise Theia seems to do well in finding the correct relative pose (compared to actual GPS and IMU measurements). But it gets stumped on localization. The images were captured in an orbital path around the scene, a similar collection taken on the same day minutes before, with a 400mm and same path, was able to reconstruct nicely. So, my assumption is, doubling the focal length to 800mm is whats causing the problem. A in-accurate initial guess for focal length could cause this, except estimate_relative_pose shows good results using given calibration data. These are the results for all 3 sfm types, Global, Incremental and Hybrid (I can share my log files if you'd like to take a look?).

First, Global:

Reconstruction estimation statistics: Num estimated views = 28 Num input views = 47 Num estimated tracks = 1576 Num input tracks = 17746 Pose estimation time = 0.060643 Triangulation time = 2.55008 Bundle Adjustment time = 7.275 Total time = 9.91193

Global Reconstruction Estimator timings: Initial view graph filtering time = 6.7e-05 Camera intrinsic calibration time = 8e-06 Rotation estimation time = 0.032925 Rotation filtering time = 0.000396 Relative translation optimization time = 0.012809 Relative translation filtering time = 0.00224 Position estimation time = 0.012273 I0429 06:41:11.449508 8231 sparse_reconstruction.cc:542] Num views: 28 Num 3D points: 1576 I0429 06:41:11.450805 8231 print_reconstruction_statistics.h:83] Num observations: 3152 Num reprojections behind camera: 0 Mean reprojection error = 0.615282 Median reprojection_error = 0.408156 I0429 06:41:11.450884 8231 print_reconstruction_statistics.h:112] Mean track length: 2 I0429 06:41:11.450893 8231 print_reconstruction_statistics.h:119] Median track length: 2 I0429 06:41:11.450907 8231 print_reconstruction_statistics.h:123] Track length histogram = [2 - 3) = 1576 [3 - 4) = 0 [4 - 5) = 0 [5 - 6) = 0 [6 - 7) = 0 [7 - 8) = 0 [8 - 9) = 0 [9 - 10) = 0 [10 - 15) = 0 [15 - 20) = 0 [20 - 25) = 0 [25 - 50) = 0

Second, Incremental:

I0429 06:51:36.625181 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625186 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625192 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625197 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625214 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625221 11959 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:51:36.625238 11959 localize_view_to_reconstruction.cc:240] Failed to localize view id 0 with only 0 out of 31 features as inliers. I0429 06:51:36.627971 11959 reconstruction_builder.cc:384] Reconstruction estimation statistics: Num estimated views = 2 Num input views = 47 Num estimated tracks = 259 Num input tracks = 18682 Pose estimation time = 0.006479 Triangulation time = 0 Bundle Adjustment time = 0 Total time = 0.406825

Incremental Reconstruction Estimator timings: Time to find an initial seed for the reconstruction: 0.148771 I0429 06:51:36.672530 11959 sparse_reconstruction.cc:542] Num views: 2 Num 3D points: 259 I0429 06:51:36.672873 11959 print_reconstruction_statistics.h:83] Num observations: 518 Num reprojections behind camera: 0 Mean reprojection error = 0.970799 Median reprojection_error = 0.745175 I0429 06:51:36.672920 11959 print_reconstruction_statistics.h:112] Mean track length: 2 I0429 06:51:36.672930 11959 print_reconstruction_statistics.h:119] Median track length: 2 I0429 06:51:36.672950 11959 print_reconstruction_statistics.h:123] Track length histogram = [2 - 3) = 259 [3 - 4) = 0 [4 - 5) = 0 [5 - 6) = 0 [6 - 7) = 0 [7 - 8) = 0 [8 - 9) = 0 [9 - 10) = 0 [10 - 15) = 0 [15 - 20) = 0 [20 - 25) = 0 [25 - 50) = 0

Last, Hybrid:

I0429 06:57:23.620947 12252 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:57:23.620959 12252 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:57:23.620965 12252 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:57:23.620970 12252 perspective_three_point.cc:202] The 3 world points are collinear! No solution for absolute pose exits. I0429 06:57:23.620985 12252 localize_view_to_reconstruction.cc:240] Failed to localize view id 7 with only 0 out of 30 features as inliers. I0429 06:57:23.623701 12252 reconstruction_builder.cc:384] Reconstruction estimation statistics: Num estimated views = 2 Num input views = 47 Num estimated tracks = 1043 Num input tracks = 18465 Pose estimation time = 0.008786 Triangulation time = 0 Bundle Adjustment time = 0 Total time = 1.34915

Hybrid Reconstruction Estimator timings: Time to find an initial seed for the reconstruction: 1.00586 I0429 06:57:23.685432 12252 sparse_reconstruction.cc:542] Num views: 2 Num 3D points: 1043 I0429 06:57:23.686564 12252 print_reconstruction_statistics.h:83] Num observations: 2086 Num reprojections behind camera: 0 Mean reprojection error = 0.539039 Median reprojection_error = 0.365783 I0429 06:57:23.686789 12252 print_reconstruction_statistics.h:112] Mean track length: 2 I0429 06:57:23.686806 12252 print_reconstruction_statistics.h:119] Median track length: 2 I0429 06:57:23.686830 12252 print_reconstruction_statistics.h:123] Track length histogram = [2 - 3) = 1043 [3 - 4) = 0 [4 - 5) = 0 [5 - 6) = 0 [6 - 7) = 0 [7 - 8) = 0 [8 - 9) = 0 [9 - 10) = 0 [10 - 15) = 0 [15 - 20) = 0 [20 - 25) = 0 [25 - 50) = 0

And this is the flag file I used for all three, except the change the reconstruction estimator :

//############## Optimize For ############### // SIMPLIFIED_GEOMETRIC | PINHOLE | PINHOLE_RADIAL_TANGENTIAL | FISHEYE | FOV | DIVISION_UNDISTORTION --camera_model=PINHOLE // FOCAL_LENGTH | ASPECT_RATIO | SKEW | PRINCIPAL_POINTS | RADIAL_DISTORTION | TANGENTIAL_DISTORTION | DISTORTION_CENTER | SCALE_FACTOR --intrinsics_to_optimize=FOCAL_LENGTH|ASPECT_RATIO

//############## Feature Extraction ############### --descriptor=SIFT --feature_density=DENSE

//############## Matching Options ############### --matching_strategy=CASCADE_HASHING --lowes_ratio=0.75 --min_num_inliers_for_valid_match=20 --max_sampson_error_for_verified_match=10 --bundle_adjust_two_view_geometry=true --keep_only_symmetric_matches=true

//############## Global Feature Extraction ############### --select_image_pairs_with_global_image_descriptor_matching=true --num_nearest_neighbors_for_global_descriptor_matching=25 --num_gmm_clusters_for_fisher_vector=16 --max_num_features_for_fisher_vector_training=1000000

//############## General SfM Options ############### --reconstruction_estimator=HYBRID --min_track_length=2 --max_track_length=50 --reconstruct_largest_connected_component=true --shared_calibration=true --only_calibrated_views=true

//############## Global SfM Options ############### --global_position_estimator=LEAST_UNSQUARED_DEVIATION --global_rotation_estimator=ROBUST_L1L2 --post_rotation_filtering_degrees=15.0 --refine_relative_translations_after_rotation_estimation=true --extract_maximal_rigid_subgraph=false --filter_relative_translations_with_1dsfm=true --position_estimation_min_num_tracks_per_view=0 --position_estimation_robust_loss_width=0.1 --refine_camera_positions_and_points_after_position_estimation=true --num_retriangulation_iterations=1

//############## Incremental SfM Options ############### --absolute_pose_reprojection_error_threshold=4 --partial_bundle_adjustment_num_views=2 --full_bundle_adjustment_growth_percent=2 --min_num_absolute_pose_inliers=20

//############## Bundle Adjustment Options ############### --bundle_adjustment_robust_loss_function=HUBER --bundle_adjustment_robust_loss_width=15 --max_reprojection_error_pixels=5

//############## Track Subsampling Options ############### --subsample_tracks_for_bundle_adjustment=false --track_subset_selection_long_track_length_threshold=10 --track_selection_image_grid_cell_size_pixels=100 --min_num_optimized_tracks_per_view=100

//############## Triangulation Options ############### --min_triangulation_angle_degrees=4 --triangulation_reprojection_error_pixels=15 --bundle_adjust_tracks=true

//############## Logging Options ############### --alsologtostderr --v=2