torchexplorer icon indicating copy to clipboard operation
torchexplorer copied to clipboard

Published 20 hours ago • verified publisher icon spfrommer

Make grad / grad norm configurable

Open spfrommer opened this issue 1 year ago • 1 comments

Right now, all i/o gradients are logged using the grad norm, and parameter gradients are logged using the raw gradient. It would be nice to make this configurable.

spfrommer avatar Nov 14 '23 18:11 spfrommer

Probably best implementation would be to have the log list take: io_vals, io_grads, io_grad_norms, param_vals, param_grads, param_grad_norms

The attach parameter should be [vals, grads, grad_norms]

spfrommer avatar Dec 18 '23 13:12 spfrommer