ColossalAI

ColossalAI copied to clipboard

[BUG]: ModuleNotFoundError: No module named 'colossalai._C.cpu_adam'

4

### 🐛 Describe the bug - 运行sh examples/train_sft.sh  - 报错信息如下： [04/19/23 15:25:30] INFO colossalai - colossalai - INFO: /home/jovyan/work/projects/Example/ColossalAI/venv/lib/python3.8/site-packages/colossalai/context/parallel_context.py:522 set_device INFO colossalai - colossalai - INFO: process rank 0...

Zengpr

[BUG]: ModuleNotFoundError: No module named 'colossalai.kernel.op_builder'

1

### 🐛 Describe the bug I was trying to run: torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy colossalai_zero2 under applications/Chat/examples, and got this error. I tried possible solutions mentioned in other previous...

lxing532

[BUG]: fp32 param and grad have different shape torch.Size([5064704]) vs torch.Size([128000]) when use lora_rank=4 at stage 1

6

### 🐛 Describe the bug Traceback (most recent call last): File "train_sft.py", line 175, in train(args) File "train_sft.py", line 146, in train train(args) File "train_sft.py", line 146, in train trainer.fit(logger=logger,...

boundles

[BUG]: save model parameters are incorrect

7

### 🐛 Describe the bug After the Llama model is trained using Lora training method, the model can be saved normally. However, Lora's model parameters were not included in the...

tianbuwei

[booster] add low level zero plugin

2

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A...

ver217

[BUG]: GeminiAdamOptimizer, can't load saved optimizer correctly

12

### 🐛 Describe the bug Hi colossalai, I am trying to use colossalai to fine-tune stable diffusion. In the code, optimizer is defined as GeminiAdamOptimizer. I used the following code...

shileims

[BUG]: ModuleNotFoundError: No module named 'colossalai.utils.model.colo_init_context'

2

### 🐛 Describe the bug CUDA_VISIBLE_DEVICES=6 python train.py Traceback (most recent call last): File "train.py", line 13, in from colossalai.utils.model.colo_init_context import ColoInitContext ModuleNotFoundError: No module named 'colossalai.utils.model.colo_init_context' ### Environment absl-py...

wccccp

gemini plugin support shard checkpoint to avoid large checkpoint files.

flybird11111

[BUG]: docker tag 0.2.x

2

### 🐛 Describe the bug **Describe the bug** docker.io/hpcaitech/colossalai:0.2.x (x > 0) report Colossal AI version 0.2.0 and contain non-release tagged code from >0.2.0 and

MEllis-github

### Discussed in https://github.com/hpcaitech/ColossalAI/discussions/3606 Originally posted by **cryoco** April 19, 2023 I've seen 2 ray implementations of PPO in this repo, [#3195 ](https://github.com/hpcaitech/ColossalAI/pull/3195) and [#3309 ](https://github.com/hpcaitech/ColossalAI/pull/3309). The former makes the...

cryoco