GraphCL Confusions with info_nce loss in gsimclr.py

Thanks for your awesome codes, but I have some questions about ./unsupervised_TU/gsimclr.py file. In the loss_cal function, the denominator will minus the positive pair loss = pos_sim / (sim_matrix.sum(dim=1) - pos_sim) which is not consistent with infonce loss $\ell_{i, j}=-\log \frac{\exp \left(\operatorname{sim}\left(\boldsymbol{z}_{i}, \boldsymbol{z}_{j}\right) / \tau\right)}{\sum_{k=1}^{N} \exp \left(\operatorname{sim}\left(\boldsymbol{z}_{i}, \boldsymbol{z}_{k}\right) / \tau\right)}$

Jun 21 '21 13:06 ZhuYun97

Hi @ZhuYun97,

Thank you for asking. Please see the in the paper https://arxiv.org/pdf/2010.13902.pdf eq. (3), and also refer to the eq. (1) in simclr paper http://proceedings.mlr.press/v119/chen20j/chen20j.pdf.

Jun 24 '21 22:06 yyou1996

Thanks for your reply. But I still have some questions. The denominator contains all pairs(both positive and negative) in simclr(eq. 1). But in graphcl(eq. 3), the denominator does not contain positive pair. I am confused why the denominator of contrastive loss in graphcl doesn't contain positive pair. Does the accuracy perform better than the contrastive loss which contains positive pair in the denominator?

Jun 29 '21 07:06 ZhuYun97

Numerically I did not find a difference.

Jun 29 '21 13:06 yyou1996