ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[BUG]: Have stable diffifusion model train support tensor or pipeline parallel?

Open qiuyang163 opened this issue 2 years ago • 1 comments

🐛 Describe the bug

when I read colosalAI parallel doc,it say: image we need modify torch.nn.Linear to col_nn.Linear. But In stable diffusion model code, I found model use torch.nn.Linear now image image image

So have stable diffusion model train supported tensor or pipeline parallel now?

Environment

No response

qiuyang163 avatar Apr 23 '23 13:04 qiuyang163

No, our implementation now only supports ZeROs & GeminiDDP.

JThh avatar Apr 24 '23 15:04 JThh