ErenChan

[BUG]: math.prod is not always available

1

### 🐛 Describe the bug Hi, I use py37 while math.prod is New in version 3.8. I suggest replace the math.prod with other func. such as： ```python from functools import...

[DOC]: a readme typo in "Run GPT With Colossal-AI"

1

### 📚 The doc issue https://github.com/hpcaitech/ColossalAI/blob/main/examples/language/gpt/titans/README.md#29 there are two different cmds and they have the same comments. # run on multiple nodes with slurm I think the first may be...

### 🐛 Describe the bug I want to use DP=2, TP=2 on 2GPUS. and the info is as bellow: ``` INFO: Distributed environment is initialized, data parallel size: 2, pipeline...

### Describe the feature I often met the problem "RuntimeError: The server socket has failed to listen on any local network address." Usually, I manually run the script several times...

[BUG]: Program is Blocked when I training GPT with PP=4 on 4GPUs in titans.

1

### 🐛 Describe the bug Hi, It works normally with PP=2 on 2GPUs. Refer to another question https://github.com/hpcaitech/ColossalAI/issues/2535 But, it seems blocked when I run with PP=4 on 4GPUs, and...

[BUG]: RuntimeError when training gpt with PP=8,TP=1,zero_setted

2

### 🐛 Describe the bug I can run with pp=8,tp=1 w/o zero strategy. myconfig is ```python # from model import GPT2_small_pipeline_hybrid from model import GPT_13b_pp1d import torch from colossalai.nn.optimizer import...