optax icon indicating copy to clipboard operation
optax copied to clipboard

Updates dtype do not need to match params dtype but only grads dtype a priori

Open vroulet opened this issue 4 months ago • 0 comments

Revise #1060 in light of the discussion of #1039. Namely, make tests ensure that dtype of grads is preserved (not necessarily the same as the dtype of params for e.g. mixed precision training). Revise the patch #1060 to see if the initialization of the dtypes in the states are too stringent or not.

vroulet avatar Oct 09 '24 16:10 vroulet