ColossalAI-Examples

ColossalAI-Examples copied to clipboard

ModuleNotFoundError: No module named 'colossal_layer_norm_cuda'

3

Dear developers, I am trying to run the bert example but I got this error, any hint to fix it? Thanks

200987299

the problems of pipeline

huxin711

[enhancement] Examplify `all_reduce()` for tensor_parallel_*

3

Tutorial [1D Tensor Parallelism](https://www.colossalai.org/docs/features/1D_tensor_parallel) mentioned the use of `all_reduce()`, but the example attached doesn't show us how to do it. Quote: > on each processor, then use an all-reduce to...

ofey404

### Describe the feature An interesting application can attract a lot of attention. https://github.com/openai/CLIP https://github.com/openai/DALL-E DALL·E 2: https://cdn.openai.com/papers/dall-e-2.pdf https://mp.weixin.qq.com/s/FN9KjuHot547fa68usS98Q

binmakeswell

### Describe the feature In most examples, there are two files, namely train with engine and trainer. The code is highly redundant in these two files and we should just...

FrankLeeeee

### Describe the feature Hi, I find that we have provided too many huge models as examples, for instance, we reshape cifar-10 to 224*224 and use ViT Huge(at least not...

Gy-Lu

'RuntimeError: CUDA error: an illegal memory access was encountered' with large batch size of GPT2-example

1

### 🐛 Describe the bug When I ran gpt2-vanilla with a batch size of 64, there was a CUDA error `RuntimeError: CUDA error: an illegal memory access was encountered`. Then...

Gy-Lu

Provide an Example for Inference

2

Need to provide an example of doing inference, this should be synced in the documentation as well.

FrankLeeeee

### 🐛 Describe the bug When running BERT sequence example with README, an exception happened. `File "train.py", line 240, in main grad_norm = grad_norm.item() AttributeErrorAttributeError: : 'float' object has no...

Wesley-Jzy

Any plan for Swin Transformer?

1

### Describe the feature Hi there! Is there any plan for Swin Transformer?

bityangke