ba error

[Bin-ze]

I do ba from a known colmap model：

from pixsfm.refine_colmap import PixSfM
from pathlib import Path

refiner = PixSfM()

path_to_input_model = ~
path_to_output_model= ~
path_to_image_dir = ~

model, _, _, = refiner.refine_reconstruction(
    path_to_output_model,
    path_to_input_model,
    path_to_image_dir,
)

but get an error：

2024/04/30 17:01:45 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /home/guozebin/work_code/SFM/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2024/04/30 17:01:46 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet'},
 'overwrite_cache': False,
 'patch_size': 16,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': False}
/home/work_code/SFM/pixel-perfect-sfm/merge_block_debug/ref_model
[2024/04/30 17:01:54 pixsfm INFO] Extracting dense features...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 919/919 [04:58<00:00,  3.08it/s]
[2024/04/30 17:06:56 pixsfm INFO] Extracting references.
100%[████████████████████] 92451/92451 [00:05, 15640.5it/s]
[2024/04/30 17:07:02 pixsfm INFO] Reference Extraction Time: 5.90986s
[2024/04/30 17:07:02 pixsfm INFO] Start feature-reference bundle adjustment.
Traceback (most recent call last):
  File "ba.py", line 14, in <module>
    path_to_image_dir,
  File "/home/work_code/SFM/pixel-perfect-sfm/pixsfm/refine_colmap.py", line 127, in refine_reconstruction
    feature_manager=feature_manager)
  File "/home/work_code/SFM/pixel-perfect-sfm/pixsfm/refine_colmap.py", line 94, in run_ba
    reconstruction, feature_manager)
  File "/home/work_code/SFM/pixel-perfect-sfm/pixsfm/bundle_adjustment/main.py", line 95, in refine_multilevel
    problem_setup)
  File "/home/work_code/SFM/pixel-perfect-sfm/pixsfm/bundle_adjustment/main.py", line 152, in refine
    solver.run(reconstruction, feature_view, references)
RuntimeError: Failed to register track element.

It's worth noting that my colmap model comes from colamp scene merging tag_scene：

colmap model_merger --input_path1 {scene_1} --input_path2 {tmp_scene2} --output_path {tag_scene}

And it is correct to perform ba on the sub-model, such as scene_1.

I checked the reconstructed data structures and they are not different, I can also open via colmap gui. My idea is to optimize the scene after the merger. How should I solve the above problems?

[Phil26AT]

This seems to be some registration bug. A quick fix would be to comment this line: https://github.com/cvg/pixel-perfect-sfm/blob/40f7c1339328b2a0c7cf71f76623fb848e0c0357/pixsfm/bundle_adjustment/src/bundle_optimizer.h#L329

[Bin-ze]

Does commenting out the line have an adverse effect on the results?