zzlgreat

【deepspeed+galore】error in deepspeed with galore

2

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 通过accelerate config设置ds zero3的时候，提示以下错误，似乎和ds有某些兼容问题。另外求教galore训练可以多卡训练的的pp/tp方案？非常感谢 deepspeed版本：deepspeed 0.12.5+2ce6bf8c llama-factory已经git pull 至最新版本 训练sh脚本和Log如下所示。 ### Expected behavior ``` accelerate...

[Bug]: RuntimeError: Unknown layout

1

### Your current environment ```text PyTorch version: 2.2.2+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 18.04.6 LTS (x86_64) GCC...