When I use Trainer with Deepspeed, the Number of trainable parameters is 0

Open noob-ctrl opened this issue 2 years ago • 1 comments

The version information is as follows:

Problem

When I use Trainer with Deepspeed, the Number of trainable parameters is 0. Like this:

And it happens when using zero3. When I use zero2, it does not have this problem.

Mar 15 '23 10:03 noob-ctrl

cc @stas00

Mar 15 '23 10:03 amyeroberts

Thank you for the report, @noob-ctrl

Please let me know if this fix works for you: https://github.com/huggingface/transformers/pull/22193

Mar 16 '23 03:03 stas00

@stas00 Hi, it works now. Thank you!

Mar 16 '23 09:03 noob-ctrl

Thank you for testing, @noob-ctrl - the PR has been merged.

Mar 18 '23 04:03 stas00