pytorch-metric-learning icon indicating copy to clipboard operation
pytorch-metric-learning copied to clipboard

Published 20 hours ago • verified publisher icon KevinMusgrave

InfoNCE errors

Open davidireland3 opened this issue 1 week ago • 3 comments

I believe there are some issues with the InfoNCE loss. After stepping through the code, the denominator is calculated only for negatives in the batch (it should be similar to SupConLoss and calculated for all items in the batch except with itself). Also there appears to be multiple positives used in the numerator, unless I am mistaken, when with InfoNCE only a single positive should be used -- I think this makes it somewhere between the 'real' InfoNCE and SupConLoss?

davidireland3 avatar Jun 24 '24 14:06 davidireland3