horde-ad
horde-ad copied to clipboard
Warn if any element of the gradient is not updated by eval or if it's zeroed
If a gradient is not updated by eval, this is either an experiment ("what happens if I remove this expression") or a typo causing unused parameter. We can detect this for vector or matrix gradients, but not for scalar gradients.
Gradients that get zeroed are more common, but may be a bad sign and in case of scalars may be an unused parameter.