Contrastive-Predictive-Coding-PyTorch icon indicating copy to clipboard operation
Contrastive-Predictive-Coding-PyTorch copied to clipboard

NCE should be calculated for each output of the GRU (and future positions) instead of only a random index

Open Ieremie opened this issue 4 years ago • 0 comments

In the implementation, a random position of the sequence is used to compute the NCE loss. The paper mentions that the GRU output at each step is used to predict 12 timesteps in the future.

"The output of the GRU at every timestep is used as the context c from which we predict 12 timesteps in the future using the contrastive loss"

Was this decision made to reduce training time?

Ieremie avatar Oct 21 '21 16:10 Ieremie