Gradient-Centralization
Gradient-Centralization copied to clipboard
incorrect keywords: memory_format
I tried to use Adam_GC, but got the errors.
Pytorch version: 1.3.0
File "Adam.py", line 82, in step
state['exp_avg'] = torch.zeros_like(p, memory_format=torch.preserve_format)
TypeError: zeros_like() received an invalid combination of arguments - got (Parameter, memory_format=torch.memory_format), but expected one of:
* (Tensor input, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
* (Tensor input, bool requires_grad)
didn't match because some of the keywords were incorrect: memory_format
Can I just delete the memory_format ?
yes. Or you can update the Pytorch version.