axon icon indicating copy to clipboard operation
axon copied to clipboard

Do not include stateful parameters in optimizer state

Open seanmor5 opened this issue 1 year ago • 0 comments

There is no need to include these because they are not a part of the optimization process, so it ends up being a waste of memory duplicating them and for stateful keys like dropout it leads to memory issues

seanmor5 avatar Jan 16 '23 23:01 seanmor5