Does the model need accumulate the gradients?

[aflyingnoob]

At line 220 in main.py, the zero_grad is outside the training stage, the gradients will be accumulated along the training data.