Question: gradient accumulate

[zwhus]

Thanks for your work! I have a question about gradient accumulate on contrastive learning. you use the code as follows:

all_teacher_image_features = torch.cat(
      [teacher_image_features]
      + gathered_teacher_image_features[:rank]
      + gathered_teacher_image_features[rank + 1 :]
  )

why not use the code

gathered_teacher_image_features[self.rank] = teacher_image_features
all_teacher_image_features = torch.cat(gathered_teacher_image_features, dim=0)