Sehyun Choi

icon indicating a website link syncdoth.github.io

icon location South Korea | Hong Kong icon location MPhil @ HKUST, CSE | AI & Machine Learning | Data Science | NLP

Results 1 issues of Sehyun Choi

ncclSystemError with multi-node multi-gpu training

## Problem Description I am trying to run multi-node distributed training with pytorch. More specifically, I am using `torchrun` as distributed launcher, with `deepspeed`. The code works fine with single-node,...