Jinghan Li comments

Repositories
Issues
Comments

Results 2 comments of


                                            Jinghan Li

DeepSpeedEngineWrapper.backward() does a bit too much

> Does that mean if I use deepspeed integration with huggingface's `Trainer`, the parameters will be updated twice in each step? no, it won't.

DeepSpeedEngineWrapper.backward() does a bit too much

https://github.com/huggingface/accelerate/pull/3819 I created this PR to debug parameters for a short time, feel free to use it. But even with it still heavily depends on the deepspeed engine.