contrastive-predictive-coding icon indicating copy to clipboard operation
contrastive-predictive-coding copied to clipboard

Published 20 hours ago • verified publisher icon davidtellez

Confusion about the batch size and negative pairs

Open Hanzy1996 opened this issue 4 years ago • 0 comments

As discussed in the original paper, the training relies on a large number of negative pairs to tighten the lower bound of the Mutual Information, which corresponds to log(N). However, in this code, the negative pairs are constructed only in a mini-batch. Why are these negative pairs enough?

Hanzy1996 avatar Jul 31 '20 05:07 Hanzy1996