how to use zero-offload?

[xdnjust]

Hi, I am trying to train a GPT-2 model using "DeepSpeed-Chat” code. Bur in step 1, when I use the "--offload", I got a error. below is the problem:

[Modas-Li]

not supported now

[jeffra]

Zero offload isn’t supported yet, we’re actively working on this though. Will update when it’s released :)

Feel free to also follow our Twitter account for updates like this: https://twitter.com/msftdeepspeed

[EikeKohl]

Hey @jeffra 🙂, is this not implemented only for the hybrid engine, i.e. would it work as expected if I disable Hybrid Engine and remove the assertion from the main.py scripts?

[zhangyanbo2007]

@jeffra Can i use Zero Zero offload?

[zhihao-chen]

@jeffra Why can't we use zero-offload? Can you tell me the specific reason? I can successfully enable zero-offload。