papers-notebook ZeRO-Offload: Democratizing Billion-Scale Model Training

ZeRO-Offload: Democratizing Billion-Scale Model Training

Open gaocegege opened this issue 3 years ago • 2 comments

https://arxiv.org/abs/2101.06840

Nov 16 '21 06:11 gaocegege

https://www.deepspeed.ai/news/2021/03/07/zero3-offload.html

Nov 18 '21 01:11 gaocegege

我的笔记：https://github.com/Jack47/hack-SysML/blob/master/papers/ZeRO-Offload.md

Nov 19 '21 02:11 Jack47