Kevin Musgrave

Does it make sense to apply CKA and CCA to embeddings?

I mean can they be used as a drop-in replacement for any existing losses, like the contrastive loss?

Ok let's leave it out for now, unless other people express interest.

> Which of course is correct, however, the default `DotProductSimilarity` itself normalizes the input vectors. Hmm, yeah it would make more sense for `DotProductSimilarity` to not normalize the vectors. That...

Thanks @stompsjo! Also if you're looking specifically for [Supervised Contrastive Learning](https://arxiv.org/abs/2004.11362) there is a loss function for that: https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#supconloss

> Firstly, I wanted to ask in the DDP example listed, is the validation (or test) dataset partition like the training data? Since it only happens when testing the model,...

Apologies for the late reply. You can use the concept of `ref_emb` to separate anchors from positives and negatives. For example, using ContrastiveLoss: ```python from pytorch_metric_learning.losses import ContrastiveLoss loss_fn =...

Thanks for bringing this up. This may be an issue with the wording in the documentation. Here are the equations from the [original paper](https://openaccess.thecvf.com/content_CVPR_2019/papers/Wang_Multi-Similarity_Loss_With_General_Pair_Weighting_for_Deep_Metric_Learning_CVPR_2019_paper.pdf). Negative pairs:  Positive pairs: ...

Do you know what settings were used to get 0.8 accuracy?

I can see how it is confusing to have "epochs" but not have the entire dataset be used in the epoch. This main point of this sampler is to give...