pytorch-lightning icon indicating copy to clipboard operation
pytorch-lightning copied to clipboard

shortcuts for logging weights and biases norms

Open heth27 opened this issue 1 year ago • 0 comments

Description & Motivation

Knowing the norm of weights was necessary to debug float16 training for me.

Pitch

from lightning.pytorch.utilities import grad_norm

norms = grad_norm(self.layer, norm_type=2)

something like this for weights would be convenient.

Alternatives

No response

Additional context

No response

cc @borda

heth27 avatar Aug 11 '24 23:08 heth27