Superpoint + superglue w/ hloc give 90 degree camera orientation error

[sfm-user-1]

Hi, thanks for making this repo and the notebooks.

I tried out the superpoint+superglue+colmap notebook here https://github.com/cvg/Hierarchical-Localization/blob/master/pipeline_SfM.ipynb

using the 35-image dataset here: http://vision.maths.lth.se/calledataset/de_guerre/de_guerre.zip

It seems superpoint+superglue w/ HLOC estimates most camera orientations incorrectly by 90 degrees error about the +z camera axis (roll). I don't see any such issue with vanila COLMAP. I wonder if you could try reproducing or investigating this result?

Thanks again!

[sfm-user-1]

Hi, a gentle bump on this. 90 degree orientation error for most cameras seems like it is a bug in the hloc source code. Thanks!

[sarlinpe]

This indeed looks like a problem, I'll investigate once I find the time. Thank you for reporting this!

[sarlinpe]

Found the culprit - PR https://github.com/cvg/Hierarchical-Localization/pull/373 solves this bug. Unfortunately SuperPoint+SuperGlue cannot match 90-degree rotated images, so for now hloc registers only 26/35 images (those in portrait mode). Fixing this is pretty high on our priority list. For now I suggest baking the EXIF rotation into the images by reading+writing them with OpenCV:

import cv2
for p in input_dir.iterdir():
    cv2.imwrite(str(output_dir / p.name), cv2.imread(str(p)))

This prevents COLMAP from sharing intrinsics between rotated images but the negative impact should be negligible.