Any plan for Lion optimizer support?

Open sjs2666 opened this issue 2 years ago • 1 comments

Do you have any plans to support Lion, an optimizer that is claimed to be more memory-efficient and perform better than Adam?

https://arxiv.org/abs/2302.06675

Feb 24 '23 07:02 sjs2666

Does anyone want to implement Lion for apex?

Dec 27 '23 10:12 SkyAndCloud