PyHessian icon indicating copy to clipboard operation
PyHessian copied to clipboard

Potential bugs

Open yxiao54 opened this issue 1 year ago • 0 comments

Line 71 of utils.py grads.append(0 if param.grad is None else param.grad + 0.) should rewrite as: grads.append(param-param if param.grad is None else param.grad + 0.)

The current implementation may cause bugs when there are unused layers in the model. To be specific, when a layer was set require_grad as true but doesn't participate in forward or backward participation, it's grad was set as float zero. It will trigger an error when torch.autograd checks the shape of grads. Detail can be seen in this discussion: https://github.com/amirgholami/PyHessian/issues/8

yxiao54 avatar Jun 07 '23 02:06 yxiao54