colmap
Pose Prior based Incremental Mapper
This PR adds the possibility to apply the Incremental Mapper pipeline while using position priors taken from the database. The prior position can be either in geographic coordinates or in cartesian coordinates. The position priors are used during the global BA only. For now, this prior based pipeline is only accessible through command line. A unit test has been added.
More is to come (add python bindings, prior based BA controller, possibility to setup priors from a file instead of through the database, python scripts to populate the database with priors, etc...) but I'd like to ensure that you are already okay with the current state of this PR before diving further.
@sarlinpe thanks for your reply! I have a sample dataset of real data here acquired with a UAV with photos having GPS in the EXIF metadata here: https://filesender.renater.fr/?s=download&token=0418e1d0-ed8d-4bd4-89ec-e9f0b7772fe7.
On the high level, this looks great to me, thank you for your efforts to implement this feature and the diligence on implementing tests, etc. I left a few suggestions as comments.
it is a fantastic feature. But i'm a little confusing now about the PosePrior class in this repo. As to my understanding, this class can only hold the Position prior that is only the Translation Vector prior known is used. and (do point out if i'm wrong) i understand your initial idea is to use the full prior pose, so where the Quaternion is hiding?
@HernandoR Very good point. The PosePrior and PosePriorBA only handle position priors for now but can be extended to orientation priors in the future. We don't have good data to develop this for now - I guess that such priors are most likely to be obtained with compass?
No tests were added for PositionPriorErrorCostFunction. Do you plan to add it in the future? : )
@B1ueber2y I wanted to do it and then forgot... I have just made a PR with test for PositionPriorErrorCostFunctor!
Yes https://github.com/colmap/colmap/blob/1a497f023153cf7f7074e692e066137342ab645e/src/pycolmap/estimators/bundle_adjustment.cc#L195-L201
May I ask for example on how to use this interface?
Until recently my flow looked like this:
1. rec = pycolmap.Reconstruction()
2. define camera instances: pycolmap.Camera
3. define images (camera poses + image paths):
* im = pycolmap.Image(image_name, [], cam_from_worldinv, camera_id, image_id)
* rec.add_image(im) (repeat for all images...)
4. Extract and Match features (with HLOC):
4.a extract_features.main(...)
4.b match_features.main(...)
5. Perform triangulation using the existing camera poses
triangulation.main(
reference_model=reconstruction_path,
sfm_dir = sfm_path,
image_dir=Path(output_dir),
pairs=sfm_pairs,
features=features_path,
matches=matches_path,
skip_geometric_verification = True
)
final_reconstruction = pycolmap.Reconstruction(sfm_path)
**pycolmap.bundle_adjustment(final_reconstruction, options)**
A) With the pycolmap.create_pose_prior_bundle_adjuster how would the above flow change? B) Is this a correct definition of a PosePrior instance?
pose_prior = pycolmap.PosePrior()
*Set the position and orientation priors
pose_prior.position = np.array([1.0, 2.0, 3.0], dtype=np.float64)
pose_prior.orientation = np.array([0.0, 0.0, 0.0, 1.0], dtype=np.float64) # Quaternion, scalar first
C) Would the pose_prior_bundle_adjuster be executed by calling "solve" and where would results be stored?
pose_prior_bundle_adjuster = pycolmap.create_pose_prior_bundle_adjuster(
options=bundle_adjustment_options,
prior_options=pose_prior_bundle_adjustment_options,
config=bundle_adjustment_config,
pose_priors=pose_priors,
reconstruction=reconstruction
)
pose_prior_bundle_adjuster.solve()
A) You would like perform a pose prior based bundle adjustment after your call to triangulation. You may want to alternative a few rounds of (re-)triangulation and bundle adjustment until convergence. B) The pose prior can currently only contain position information. You also have to define the coordinate system convention for each pose prior. C) The results will be stored directly in the reconstruction.
@ferreram
Thanks for your great code!
I tried the pose_prior_mapper cmd, but I got an error saying No pose priors in database...
I performed the following processes:
- get images and poses from ARKit app (without points clouds)
- convert ARKit fmt to colmap fmt (images.bin/txt, cameras.bin/txt, points3D.bin/txt(empty), database.db)
directory configuration:
arkit_data_path/
|--- input/
|--- frame_00001.jpg
|--- frame_....jpg
|--- sparse/
|--- 0/
|--- images.bin/txt
|--- cameras.bin/txt
|--- points3D.bin/txt
|--- database.db
- extract feature
colmap feature_extractor
--database_path arkit_data_path/sparse/0/database.db
--image_path arkit_data_path/input
--SiftExtraction.use_gpu 1
--ImageReader.single_camera 1
- feature matching
colmap sequential_mathcer
--database_path arkit_data_path/sparse/0/databse.db
--SiftExtraction.use_gpu 1
- pose prior mapper
colmap pose_prior_mapper
--image_path arkit_data_path/input
--database_path arkit_data_path/sparse/0/database.db
--output_path sfm_processed_data_path
--Mapper.multiple_models 0
--Mapper.ba_refine_focal_length 0
--Mapper.ba_refine_extra_params 0
--overwrite_priors_covariance 1
--prior_position_std_x 0.05
--prior_position_std_y 0.05
--prior_position_std_z 0.01
- bundle adjuster
colmap bundle_adjuster
--input_path sfm_processed_data_path
--output_path sfm_processed_data_path
--BundleAdjustment.refine_focal_length 0
--BundleAdjustment.refine_principal_point 0
--BundleAdjustment.refine_extra_params 0
Error Encountered:
I1202 01:23:10.560431 948 sfm.cc:54] Setting up database pose priors with the same covariance matrix:
0.0025 0 0
0 0.0025 0
0 0 0.0001
I1202 01:23:10.574162 948 incremental_pipeline.cc:237] Loading database
I1202 01:23:10.624039 948 database_cache.cc:66] Loading cameras...
I1202 01:23:10.628621 948 database_cache.cc:76] 1 in 0.005s
I1202 01:23:10.628628 948 database_cache.cc:84] Loading matches...
I1202 01:23:11.159878 948 database_cache.cc:89] 1082 in 0.531s
I1202 01:23:11.159904 948 database_cache.cc:105] Loading images...
I1202 01:23:16.291671 948 database_cache.cc:153] 200 in 5.132s (connected 200)
I1202 01:23:16.291709 948 database_cache.cc:164] Loading pose priors...
I1202 01:23:16.612479 948 database_cache.cc:175] 0 in 0.321s
I1202 01:23:16.612496 948 database_cache.cc:184] Building correspondence graph...
I1202 01:23:16.776397 948 database_cache.cc:210] in 0.164s (ignored 0)
I1202 01:23:16.776595 948 timer.cc:91] Elapsed time: 0.103
I1202 01:23:16.776602 948 database_cache.cc:228] Setting up prior positions...
E1202 01:23:16.776602 948 database_cache.cc:234] No pose priors in database...
E1202 01:23:16.950085 948 sfm.cc:465] failed to create sparse model
If you have any solutions., please let me know!
To my knowledge, you have to insert the pose prior to the database, ie. the database.db. Colmap itself can only read and convert that info from exif info of the images. Please check if your images have corresponding information. Otherwise you have to insert manually.
I think there is a py interface named writeposeprior, please checkout on the pycolmap directory.
Liu Zhen
From: Shuta Ochiai @.> Sent: Monday, December 2, 2024 10:38:55 AM To: colmap/colmap @.> Cc: Liu Zhen @.>; Mention @.> Subject: Re: [colmap/colmap] Pose Prior based Incremental Mapper (PR #2660)
@ferreramhttps://github.com/ferreram Thanks for your great code! I tried the pose_prior_mapper cmd, but I got an error saying No pose priors in database...
I performed the following processes:
- get images and poses from ARKit app (without points clouds)
- convert ARKit fmt to colmap fmt (images.bin/txt, cameras.bin/txt, points3D.bin/txt(empty), database.db)
directory configuration: arkit_data_path/ |--- input/ |--- frame_00001.jpg |--- frame_....jpg |--- sparse/ |--- 0/ |--- images.bin/txt |--- cameras.bin/txt |--- points3D.bin/txt |--- database.db
- extract feature
colmap feature_extractor --database_path arkit_data_path/sparse/0/database.db --image_path arkit_data_path/input --SiftExtraction.use_gpu 1 --ImageReader.single_camera 1
- feature matching
colmap sequential_mathcer --database_path arkit_data_path/sparse/0/databse.db --SiftExtraction.use_gpu 1
- pose prior mapper
colmap pose_prior_mapper --image_path arkit_data_path/input --database_path arkit_data_path/sparse/0/database.db --output_path sfm_processed_data_path --Mapper.multiple_models 0 --Mapper.ba_refine_focal_length 0 --Mapper.ba_refine_extra_params 0 --overwrite_priors_covariance 1 --prior_position_std_x 0.05 --prior_position_std_y 0.05 --prior_position_std_z 0.01
- bundle adjuster
colmap bundle_adjuster --input_path sfm_processed_data_path --output_path sfm_processed_data_path --BundleAdjustment.refine_focal_length 0 --BundleAdjustment.refine_principal_point 0 --BundleAdjustment.refine_extra_params 0
Error Encountered:
I1202 01:23:10.560431 948 sfm.cc:54] Setting up database pose priors with the same covariance matrix: 0.0025 0 0 0 0.0025 0 0 0 0.0001 I1202 01:23:10.574162 948 incremental_pipeline.cc:237] Loading database I1202 01:23:10.624039 948 database_cache.cc:66] Loading cameras... I1202 01:23:10.628621 948 database_cache.cc:76] 1 in 0.005s I1202 01:23:10.628628 948 database_cache.cc:84] Loading matches... I1202 01:23:11.159878 948 database_cache.cc:89] 1082 in 0.531s I1202 01:23:11.159904 948 database_cache.cc:105] Loading images... I1202 01:23:16.291671 948 database_cache.cc:153] 200 in 5.132s (connected 200) I1202 01:23:16.291709 948 database_cache.cc:164] Loading pose priors... I1202 01:23:16.612479 948 database_cache.cc:175] 0 in 0.321s I1202 01:23:16.612496 948 database_cache.cc:184] Building correspondence graph... I1202 01:23:16.776397 948 database_cache.cc:210] in 0.164s (ignored 0) I1202 01:23:16.776595 948 timer.cc:91] Elapsed time: 0.103 I1202 01:23:16.776602 948 database_cache.cc:228] Setting up prior positions... E1202 01:23:16.776602 948 database_cache.cc:234] No pose priors in database... E1202 01:23:16.950085 948 sfm.cc:465] failed to create sparse model
If you have any solutions., please let me know!
— Reply to this email directly, view it on GitHubhttps://github.com/colmap/colmap/pull/2660#issuecomment-2510443836, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK4XSWCYXGHHHCF6BROZINT2DPB37AVCNFSM6AAAAABLA434QSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJQGQ2DGOBTGY. You are receiving this because you were mentioned.Message ID: @.***>
@ahojnnes I tried a basic call to create_pose_prior_bundle_adjuster, but get an exception:
rec = pycolmap.Reconstruction('output/sfm') #Folder containing images.bin, cameras.bin and points3D.bin
pose_prior_bundle_adjuster = pycolmap.create_pose_prior_bundle_adjuster(
pose_priors=pose_priors,
reconstruction= rec
)
pose_priors is a dict of int (image id) VS pycolmap.PosePrior objects
create_pose_prior_bundle_adjuster(): incompatible function arguments. The following argument types are supported:
1. (options: pycolmap._core.BundleAdjustmentOptions, prior_options: pycolmap._core.PosePriorBundleAdjustmentOptions, config: pycolmap._core.BundleAdjustmentConfig, pose_priors: dict[int, pycolmap._core.PosePrior], reconstruction: pycolmap._core.Reconstruction) -> pycolmap._core.BundleAdjuster
Invoked with: kwargs: options=BundleAdjustmentOptions(loss_function_type=LossFunctionType.TRIVIAL, loss_function_scale=1.0, refine_focal_length=False, refine_principal_point=False, refine_extra_params=False, refine_extrinsics=True, print_summary=True, use_gpu=True, gpu_index='-1', min_num_residuals_for_cpu_multi_threading=50000, min_num_images_gpu_solver=50, max_num_images_direct_dense_cpu_solver=50, max_num_images_direct_sparse_cpu_solver=1000, max_num_images_direct_dense_gpu_solver=200, max_num_images_direct_sparse_gpu_solver=4000), pose_priors={1: PosePrior(position=[43.502, -30.3787, -114.512], position_covariance=[1, 0, 0, 0, 1, 0, 0, 0, 1], coordinate_system=CARTESIAN), 2: PosePrior(position=[43.4313, -30.4905, -114.934], position_covariance=[1, 0, 0, 0, 1, 0, 0, 0, 1], coordinate_system=CARTESIAN), 3: PosePrior(position=[43.5459, -30.594, -115.323], position_covariance=[1, 0, 0, 0, 1, 0, 0, 0, 1], ....
Could you share a mini example that could serve me and the community as a guide?
Version details:
COLMAP_build: str = "Commit e778be63 on 2024-11-29 with CUDA"
COLMAP_version: str = "COLMAP 3.12.0.dev0"
__ceres_version__: str = "2.3.0"
__version__: str = "3.12.0"
Is there an official instruction or example to use the GPS prior? That will be very useful for the community.
