mup icon indicating copy to clipboard operation
mup copied to clipboard

add width_mult to optimizer dict

Open marcobellagente93 opened this issue 1 year ago • 0 comments

Adding the width_mult key to the MuAdam state dictionary to make it more easy to use the class, e.g. to enable its correct use in https://github.com/EleutherAI/gpt-neox

marcobellagente93 avatar Oct 19 '23 14:10 marcobellagente93