Is it possible to implement Accelerate from Hugging Face for training?

[Omegadarling]

Is there an existing issue for this?

• [x] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Hugging Face created a library for training models with multiple GPUs and I have a few machines with multiple GPUs, including one with 10 x 3090s so I'd love to leverage all that power.

Proposed workflow

1. Go to Train tab
2. Check a box that says "use Accelerate library"
3. Waste less time

Additional information

Here's the link to Accelerate: https://huggingface.co/docs/transformers/accelerate

[wfjsw]

I thought it is there already?

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/066390eb5683945a6e094a817584ada6b1f7118e/webui.sh#L158-L163

[ClashSAN]

accelerate was added as an optional toggle for dreambooth extension, not for textual inversion or hypernetwork I believe.

[Omegadarling]

@ClashSAN

accelerate was added as an optional toggle for dreambooth extension, not for textual inversion or hypernetwork I believe.

I use Dreambooth all the time and I've never seen that option. Is it something in the script but not in the GUI?

[ClashSAN]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/0d098e46568a268180974953fe2783fff1333040

Is it something in the script but not in the GUI?

that's likely. diffusers library will usually ask if you want to run with accelerate option. it should just work once installed, but its better to confirm on their extensions page.

I don't know the answer because I haven't used dreambooth extension

[Omegadarling]

set ACCELERATE="True" in webui-user.bat doesn't install the necessary stuff automatically. I did manually install it and create the config file, but all kinds of WARNING: Ignoring invalid distribution -celerate (c:\stable-diffusion-webui\venv\lib\site-packages) when I try to run Accelerate. It works again if I remove the set ACCELERATE="True" , so what's the deal with Accelerate?

Here's a copy of what happens after I add set ACCELERATE="True" log.txt

[Omegadarling]

I'm not sure what I need to fix to get Accelerate to work. Here's another log, possibly with more info, possibly pretty much the same. Is there a WIKI to explain the proper way to use Accelerate with Automatic1111? SD_accelerateTrue_log_2023-01-11-1743.txt

[Omegadarling]

I decided to run pip install -r .\requirements.txt and try running with set ACCELERATE="True" again and now I'm getting new errors, and yet somehow it seems like this is progress. Here's my powershell log: SD_accelerateTrue_log_2023-01-11-1823.txt

I can't tell what matters, but the first error it throws is NOTE: Redirects are currently not supported in Windows or MacOs.