AffNet features & patch extraction

[ducha-aiki]

Hi,

I have happily noticed that you are using kornia :)

1. Could we be even more useful for you? I have recently created a wrapper around OpenCV detectors into kornia_moons, so one can easily combine them with modern patch descriptors, or even AffNet affine shape, like this example, which combines AKAZE features with AffNet affine shape and then describes them with HyNet descriptor.

import cv2
import kornia.feature as KF
from kornia_moons.feature import *
detector = OpenCVDetectorWithAffNetKornia(cv2.AKAZE_create(threshold=3e-3))
feature = KF.LocalFeature(detector, KF.LAFDescriptor(KF.HyNet(True)))

lafs, resps, desc = feature(timg)

Here is a result of such matching

2. Is there any reason to use floor instead of round? If you observe that it leads to the better results, we can fix it directly in kornia

https://github.com/cvg/Hierarchical-Localization/blob/master/hloc/extractors/dog.py#L33

[sarlinpe]

Hi Dmytro,

It seems that we share the same goal of providing a standard interface for feature extraction and matching :)

1. We are pretty happy that kornia now provides GPU-accelerated patch extraction and patch descriptors - we care a lot about speed (now we run hloc on large datasets) so this is very helpful. Interfacing with OpenCV is cool, maybe that will come handy later on. One major remaining pain point is SIFT - for now we use the VLFeat implementation, but it's too slow for large images. It would be great if kornia's DoG detector and SIFT descriptor could match its accuracy. I am aware of this PR but know that a good implementation of SIFT requires a ton of work so I can't afford to look into this myself.

2. I was getting a lower performance when using kornia.feature.laf.extract_patches_from_pyramid instead of your extract_patches package, so I looked for differences and found out that floor vs round was the only one:

# extract_patches.core.convert_xyA/convert_cv2_keypoints
pyr_idx = max(0, int(math.log(s,2)))
# vs kornia.feature.laf.extract_patches_from_pyramid
pyr_idx = (scale.log2() + half).relu().long()

My intuition is that round over-blurs patches while floor retains some sharpness. Happy to send a PR if deemed useful.

cc @mihaidusmanu

[ducha-aiki]

@Skydes PR is not needed, I wanted confirmation :) https://github.com/kornia/kornia/pull/1616

[ducha-aiki]

@Skydes since the thing was fixed in kornia, how about removing code from hloc and rely on kornia instead? I can make a PR if needed.

[sarlinpe]

Thanks for the reminder, done in https://github.com/cvg/Hierarchical-Localization/pull/216