sreetamasarkar

CUDA memory increases after each loss.backward()

6

I am trying to use the FMoE layer in a ViT-Base model for a simple classification task. However, there is a gradual increase in CUDA memory, which eventually leads to...