EfficientNet-PyTorch
EfficientNet-PyTorch copied to clipboard
lukemelas
Feature vector for image similarity measure
For image similarity measurement, we tried to get the feature vectors using the method model.extract_features(img). However, comparing to nasnet_large for example, not only the feature vector from the EfficientNet is huge (87808 dimension total, 1792x7x7), the L2 distance of two such huge vectors from two similar images doesn't reflect the similarity as well as that from feature vectors of nasnet_large (only about 4000 dimension). Is there any other way to get feature vectors , maybe from a difference layer, that can be better suited for similarity measure of two images? Thanks!
That feature vector still has spacial information, so you might try averaging over the spacial dimensions (e.g. use torch.mean to make the 1792x7x7-dim vector to a 1792-dim vector). Let me know if this works any better!
yes, averaging works. However, I am not sure if the feature vectors obtained this way are orientational invariant. Will do some testing...
Great! Keep me updated.
On Mon, Aug 26, 2019, 8:53 PM drjtan [email protected] wrote:
yes, averaging works. However, I am not sure if the feature vectors obtained this way are orientational invariant. Will do some testing...
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lukemelas/EfficientNet-PyTorch/issues/68?email_source=notifications&email_token=ADFQ4MG5FMKSUXVABSRQGFTQGR3J5A5CNFSM4IPK2L22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5GDXQY#issuecomment-525089731, or mute the thread https://github.com/notifications/unsubscribe-auth/ADFQ4MA5BRMLH3E7BDYW2HLQGR3J5ANCNFSM4IPK2L2Q .
yes, averaging works. However, I am not sure if the feature vectors obtained this way are orientational invariant. Will do some testing...
Can you share your test results? I am also working on image similarity. thank you very much!
Not orientationally invariant, unfortunately. In our application, we are interested in the dl feature of the object in the image, and what we did was to apply a PCA to align the object to a pre-determined orientation, that seems to solve the problem.
how did you use PCA to align to a pre-determined orientation? I am currently tackling the same problem, I built an autoencoder out of efficientnet_b7, the encoder (effnet) extracts the features and the decoder (effnet transposed) reconstructs the image to compute the loss and teach the encoder what the best features are to represent an image, after the encoder is trained I use it to extract features of an image database and a single query image, and then use NearestNeighbor algo to search for similar images, and I too was failing with getting similar images, was getting better results computing hashes(wavelet hash, dhash, etc.)
in our application, the objects of interest are all rigid objects, which means they have well defined contours. Aligning contour using PCA is quick and easy.