BatchFormer icon indicating copy to clipboard operation
BatchFormer copied to clipboard

The loss of BatchFormer with MoCo-v3 ?

Open iorange-77 opened this issue 1 year ago • 2 comments

May I ask the 133-134 lines of code in ./moco-v3/moco/builder.py: loss = self.contrastive_loss(q1[:N], k2[:N]) + self.contrastive_loss(q1[:N], k2[N:]) + self.contrastive_loss(q1[N:], k2[:N]) + self.contrastive_loss(q1[N:], k2[:N]) +
self.contrastive_loss(q2[:N], k1[:N]) + self.contrastive_loss(q2[:N], k1[N:]) + self.contrastive_loss(q2[:N], k1[:N]) + self.contrastive_loss(q2[:N], k1[N:])

Is the loss calculated incorrectly? should be: ?? loss = self.contrastive_loss(q1[:N], k2[:N]) + self.contrastive_loss(q1[:N], k2[N:]) + self.contrastive_loss(q1[N:], k2[:N]) + self.contrastive_loss(q1[N:], k2[N:]) +
self.contrastive_loss(q2[:N], k1[:N]) + self.contrastive_loss(q2[:N], k1[N:]) + self.contrastive_loss(q2[N:], k1[:N]) + self.contrastive_loss(q2[N:], k1[N:])

If my understanding is wrong, what does this loss calculation method mean?

iorange-77 avatar Nov 30 '22 08:11 iorange-77