hanklin

icon location MS student from National Taiwan University.

Results 3 comments of hanklin

最新的代码，deepspeed (0.14.2)4*A40 采用deepspeed zero3_offload，调试Qwen110B，系统报错：RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu

嗨，我在 tune llama-2 的時候遇到了一樣的問題，我把 deepspeed downgrade 到 0.14.0 就可以跑了！

Is DDP enabled for speaker identification using wav2vec2?

Hi, I solve it by downgrade my torch version from 2.0.1 to 1.13.0. You can refer to this page to find the instructions to install the older version of torch/torchaudio....

Port problem when using DDP (more than 2 experiments on same server)

Hi, you can specified the argument `--master-port` after `--nproc_per_node`. Since you already launch a DDP training, the default port `29500` is utilized by it and thus the error occurs. You...