[deepspeed] offload + non-cpuadam optimizer exception doc

Open stas00 opened this issue 2 years ago • 1 comments

part 2 of https://github.com/huggingface/transformers/pull/22043, but we can't merge it until deepspeed==0.8.3 is released.

This PR documents the new feature and up's the min deepspeed version.

XXX: DO NOT MERGE UNTIL deepspeed==0.8.3 is released.

I'm keeping it as a DRAFT so that I don't mistakenly merge it to soon. But we can pre-approve.

cc: @jeffra

Mar 09 '23 03:03 stas00

The documentation is not available anymore as the PR was closed or merged.

Mar 09 '23 03:03 HuggingFaceDocBuilderDev