Zach Mueller

@BenjaminBossan will go live in v4.41.0 and stay until 4.46.0 as detailed in the thread (6 months)

And yes, this PR is open for the future :)

What kind of GPUs are these?

You want to look at things that happen when the `gradient_synchronization` is enabled in the GradientState. You can also see tests in the test_script.py and test_synchronization.py scripts to find our...

This has now been introduced in https://github.com/huggingface/accelerate/pull/1049. Please follow the new `accelerate config` command to set this up. Below are some directions: 1. Install accelerate via `pip install git+https://github.com/huggingface/accelerate (and...

Thanks for the bump all, will look at it this week

Hi @jianguoz, apologize this has taken a month to get to I understand that can be quite frustrating 😢 I was indeed able to recreate your issue on a new...

@jianguoz I did it from a fresh instance when I posted those instructions and did not face any issues. Is it still that "fail to execute" issue?

As the `Trainer` dropped fairscale support, we will not be doing so in Accelerate :)

Yes, all models being trained should be passed to `accumulate`