Supervised Contrastive Loss

Open 0x1orz opened this issue 1 year ago • 2 comments

Add the Supervised Contrastive Loss ref. Supervised Contrastive Learning ?

Jun 06 '24 03:06 0x1orz

If you would like to take it on @KylevdLangemheen you need to comment, so I can assign it to you!

I looked at this PyTorch implementation – which seems to be even the official one – and the paper. I don't think there's a benefit (or a nice way) to build the loss on top of the existing NTXentLoss, so I would probably implement it from scratch. The loss in the mentioned repository does not seem to support multi-GPU settings (or at least it does not seem to calculate contrast across devices), so that could be a really nice addition from our side. Check the NTXentLoss implementation on how we did this!

Aug 05 '25 20:08 liopeer

Ah yes, I would be interested in taking this one please!

Aug 05 '25 20:08 KylevdLangemheen