Han Jiang comments

Repositories
Issues
Comments

Results 2 comments of


                                            Han Jiang

Multi-GPU Training

![image](https://github.com/artidoro/qlora/assets/5325806/da3e8819-3004-49c1-8f0a-cbaba9bff5a4) I have 4 GPUs, why is always only one or two of them computing?

"RuntimeError: The size of tensor a (5120) must match the size of tensor b (20480) at non-singleton dimension 0" in step3

get a new error with newest master: ``` 192.168.1.51: Traceback (most recent call last): 192.168.1.51: File "/mnt/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/main.py", line 521, in 192.168.1.51: main() 192.168.1.51: File "/mnt/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/main.py", line 429, in main 192.168.1.51:...