ColossalAI-Examples issues

Overflow in GPT examples

1

### 🐛 Describe the bug I met overflow using the official scripts for GPT2. Is that a normal case? ``` cd XXX/ColossalAI/examples/language/gpt export DATA=/data/scratch/gpt_data/small-gpt-dataset.json torchrun --standalone --nproc_per_node=1 train_gpt.py --config=gpt2_configs/gpt2_zero3.py --from_torch...

feifeibear

ImportError: cannot import name 'colo_state_dict' from 'colossalai.utils.model.colo_init_context'

1

### 🐛 Describe the bug I am trying example `colo_vit` but got this error ``` Traceback (most recent call last): File "", line 1, in ImportError: cannot import name 'colo_state_dict'...

fuhengwu2021

BERT Data Preprocessing

3

### 🐛 Describe the bug NVIDIA DeepLearningExamples removed LDDL from DLE tools on Aug 16, 2022. Therefore, the guide on https://github.com/hpcaitech/ColossalAI-Examples/tree/main/language/bert/preprocessing fails to work in the following aspects: 1. pip...

JizeZhangPsych

Cannot find the gradient handler example

1

### 📚 The doc issue We provide a [runnable example](https://github.com/hpcaitech/ColossalAI-Examples/tree/main/features/gradient_handler) to demonstrate the use of gradient handler. In this example, we used DataParallelGradientHandler instead of PyTorch DistributedDataParallel for data parallel...

DarrenYing

connection failure

2

### 🐛 Describe the bug I found a runtime error while running the code: The client socket has failed to connect to any network address of (hcp-bb-03, 52873). The client...

lhj-git