colmap
Add ALIKED/LightGlue feature based on libtorch
- Adds support for extracting ALIKED features.
- Adds support for matching ALIKED features using simple dot-product based nearest neighbor search.
- Adds support for matching ALIKED and SIFT using LightGlue.
@MrNeRF We found two necessary fixes that you may want to also apply to your upstream version: https://github.com/colmap/colmap/pull/3068/commits/a6c360a33b6255682fef4778990311b6d0cb6c18 and https://github.com/colmap/colmap/pull/3068/commits/1bd573c55fb4a44112453e2ec99b804addce7d9d
@MrNeRF Do you have an example for how you converted the original lightglue .pth files to your .pt files? I am trying to also convert the SIFT model but cannot get it to be in the same format. Thanks.
@MrNeRF Do you have an example for how you converted the original lightglue .pth files to your .pt files? I am trying to also convert the SIFT model but cannot get it to be in the same format. Thanks.
Try
from collections import OrderedDict
import torch
modelname = "sift_lightglue"
filename = modelname + ".pth"
checkpoint = torch.load(filename, map_location="cpu")
# Convert to a plain dict
if isinstance(checkpoint, OrderedDict):
checkpoint = dict(checkpoint)
print(type(checkpoint)) # Now <class 'dict'>
# Save the result as a standard dict
torch.save(checkpoint, modelname + ".pt")
@MrNeRF Do you have an example for how you converted the original lightglue .pth files to your .pt files? I am trying to also convert the SIFT model but cannot get it to be in the same format. Thanks.
Try
from collections import OrderedDict import torch modelname = "sift_lightglue" filename = modelname + ".pth" checkpoint = torch.load(filename, map_location="cpu") # Convert to a plain dict if isinstance(checkpoint, OrderedDict): checkpoint = dict(checkpoint) print(type(checkpoint)) # Now <class 'dict'> # Save the result as a standard dict torch.save(checkpoint, modelname + ".pt")
Yeah! Beside heavily using the C++ debugger and asserts, I didn't do anything smart or fancy. Basically just this.
@MrNeRF Do you have an example for how you converted the original lightglue .pth files to your .pt files? I am trying to also convert the SIFT model but cannot get it to be in the same format. Thanks.
Try
from collections import OrderedDict import torch modelname = "sift_lightglue" filename = modelname + ".pth" checkpoint = torch.load(filename, map_location="cpu") # Convert to a plain dict if isinstance(checkpoint, OrderedDict): checkpoint = dict(checkpoint) print(type(checkpoint)) # Now <class 'dict'> # Save the result as a standard dict torch.save(checkpoint, modelname + ".pt")Yeah! Beside heavily using the C++ debugger and asserts, I didn't do anything smart or fancy. Basically just this.
Thank you.
It seems LiftFeat is approximately twice as fast as ALIKED and also achieves higher accuracy. It's also designed to handle challenging scenarios such as drastic lighting changes, low-texture regions, and repetitive patterns. Maybe consider switching from ALIKED to LiftFeat or adding both?
I'm really interested in this, Looking forward to see this PR merged soon! Are there any problem blocking this PR?
Colmap ALIKED + Lightglue produces worse result than hloc.
HLOC ALIKED + Lightglue with default parameters
colmap ALIKED + Lightglue with default parameters（not all images registered）
colmap ALIKED + Lightglue with --ALIKEDExtraction.max_image_size 1024 --ALIKEDMatching.min_similarity 0.1
@xiemeilong Thanks for the feedback. There are still some known issues related to torch under Windows that I cannot root cause. I am not sure if these hint at some larger issues also on other platforms. On which platform did you run this comparison? Unrelated to these issues, the current colmap implementation does not auto-rotate the images to be upright (e.g., based on EXIF image orientation information). ALIKED/LightGlue only work reliably up to ~45deg rotation change between images. Can it be that your images are inconsistently rotated?
@ahojnnes I tested this on Linux, and all images orientations are same. I found that LightGlue doesn't work well with aliked-n32 in my tests, hloc uses aliked-n16 by default.