About gradient accumulation

[yijicheng]

Thanks for your excellent work!

I am wonder why the accumulation steps are just multiplied on the one step loss, but not accumulate gradient with multiple steps.

https://github.com/autonomousvision/stylegan-t/blob/36ab80ce76237fefe03e65e9b3161c040ae888e3/training/training_loop.py#L186