horde-ad icon indicating copy to clipboard operation
horde-ad copied to clipboard

Warn if any element of the gradient is not updated by eval or if it's zeroed

Open Mikolaj opened this issue 3 years ago • 0 comments

If a gradient is not updated by eval, this is either an experiment ("what happens if I remove this expression") or a typo causing unused parameter. We can detect this for vector or matrix gradients, but not for scalar gradients.

Gradients that get zeroed are more common, but may be a bad sign and in case of scalars may be an unused parameter.

Mikolaj avatar Feb 03 '22 21:02 Mikolaj