Convergence guarantees for Lion

[xnchxy]

Thank you so much for your great implementations! My collabrators and I have recently onlined a manuscript (available at https://arxiv.org/abs/2307.10053) that provides convergence guarantees for Lion optimizer, especially in the training of nonsmooth neural networks. Moreover, we present some of our implementations at our repo (available at https://github.com/xnchxy/GeneralSGD). Hope our results could help to improve the Lion optimizer.

Best regards,

Nachuan