SimCLR
SimCLR copied to clipboard
NT_Xent Loss function: all negatives are not being used in multi-gpu case ?
@sthalles Hello, i saw there have the similar issue before, the problem is i haven't saw any torch.gather API to collect the negative terms crossing all gpu in loss function calculation. So, is that means this repo does not support the multi-gpu yet ?
Any suggestion will be appreciated !!