InvokeAI
InvokeAI copied to clipboard
invoke-ai
[enhancement]: Multiple GPU support
Is there an existing issue for this?
- [x] I have searched the existing issues
Contact Details
No response
What should this feature add?
Successful attempts have been made by @lstein with prior versions of Invoke 3.x and 4.x. There was a big change to the model manager that have put his efforts back to square 1 as @lstein mentioned as a blocker to move ahead and port his work to 5.x. That said, I remain optimistic for emerging interests in such features.
Note that support for multiple GPU could means different approaches:
- 1 generation per GPU https://github.com/neuratech-ai/ComfyUI-MultiGPU
- 1 generation split across multi-GPU operation (loading model on 1 GPU and process on another for instance) https://deepwiki.com/heurist-network/comfyui-miner-release/4.2-scaling-with-multiple-gpus https://www.runcomfy.com/comfyui-nodes/ComfyUI-MultiGPU
@lstein elegantly mentioned that he’ll be glad to share his previous work AKA Invoke MGPU (4.2 fork) to help developers in undertaking this endeavor: https://github.com/lstein/InvokeAI-MGPU/releases/tag/MGPU-v0.0.9
I'd be equally glad to spearhead testing and give access to a remote multi-GPU rig if needed.
Alternatives
No response
Additional Content
No response
Distributed inference: https://huggingface.co/docs/accelerate/usage_guides/distributed_inference#distributed-inference
Multi-GPU support: Automatically spreads your tasks across multiple GPUs: https://github.com/easydiffusion/easydiffusion
The "Swarm" name is in reference to the original key function of the UI: enabling a 'swarm' of GPUs to all generate images for the same user at once (especially for large grid generations): https://github.com/Stability-AI/StableSwarmUI
I cannot select GPUs anymore. I have one GPU (cuda=0) running for training and I would like to use a second (cuda=1) GPU for inference. It looks like the cuda device is hardcoded to 0.
