pytorch-lightning
pytorch-lightning copied to clipboard
shortcuts for logging weights and biases norms
Description & Motivation
Knowing the norm of weights was necessary to debug float16 training for me.
Pitch
from lightning.pytorch.utilities import grad_norm
norms = grad_norm(self.layer, norm_type=2)
something like this for weights would be convenient.
Alternatives
No response
Additional context
No response
cc @borda