Xingchen Song(宋星辰)

@kisseternity could u try stage-3 to double-check this issue?

double-check that there has a pretty serious question in stage-2. stage-1 & stage-3 look good and almost equal in loss & grad_norm. cc @tjruwase @loadams 

> > @kisseternity could u try stage-3 to double-check this issue? > > I've tried ZeRO3 and it's fine. Besides the training speed is fine compared to ZeRO2 with communication...

> > > > @kisseternity could u try stage-3 to double-check this issue? > > > > > > > > > I've tried ZeRO3 and it's fine. Besides the...

Hi, teams, any update?

先merge一下main

有paper link的话可以贴一下

咋样啦，有最终结果了不

hi teams, any updates？

met same issue with fp16 mode