Another usage of LightGlue

[bartoszptak]

Hi, I trained the neural network that provides (x,y) coordinates of objects and descriptors for them (such as SuperPoint).

When I use GT points and LightGlue, the matching works excellently. But when I use (x, y) estimations, LightGlue strongly filters matches (for my example, I have 264 detected points, when LightGlue returns 110 matches).

It is worth highlighting that the objects between frames can lightly move.

I set the parameters:

depth_confidence: -1
width_confidence': -1
filter_threshold': 1e-5, 

It is possible to extend the algorithm for this purpose? Could you suggest any changes/improvements for such a task?

[sarlinpe]

• Each model of LightGlue is trained for a given type of descriptors only. Using LightGlue with your own descriptors requires retraining it. We do not provide the training code but will do so soon.
• The models that we released are trained on static scenes (MegaDepth) and are thus likely to reject matches between dynamic objects, even if they move only slightly. You might need to retrain with your won data for optimal performance.

[bartoszptak]

Thanks for your answer!

• So, did you train your own descriptors of SuperPoint?
• I know that LighGlue was trained on static scenes, but I thought some of the reparametrisation was possible to adapt the algorithm.

I can't wait for the training code! Good job!