net.zero_grad()

[jason-engage]

Hi, I'd like to try this in my u2net imagenet model.

Can i simply drop it in and replace Adam/SGD?

I noticed that you are calling net.zero_grad() in your training loop, however in my script its optimizer.zero_grad()... will this still work with AGD set as the optimizer?

Any other suggestions on getting it to work correctly? Thanks for putting another idea out there, Cheers