ColossalAI

ColossalAI copied to clipboard

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

yuanheng-zhao

### Is there an existing issue for this bug? - [X] I have searched the existing issues ### 🐛 Describe the bug Got `TypeError: LlamaInferenceForwards.llama_causal_lm_forward() got an unexpected keyword argument...

hiprince

[BUG]: No module named 'dropout_layer_norm'

2

### Is there an existing issue for this bug? - [x] #5795 ### 🐛 Describe the bug ModuleNotFoundError: No module named 'dropout_layer_norm' [2024-05-17 03:23:11,932] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank:...

apachemycat

[gemini] async grad chunk reduce (all-reduce&reduce-scatter)

1

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A...

botbw

[BUG]: exception: Encountered a bad command exit code!

1

### 🐛 Describe the bug 运行llama示例, bash batch8_seq512.sh 报错 exception: Encountered a bad command exit code! 运行gpt示例，单机可成功，使用colossal run 多机同样报以上错误。 (PyTorch-2.0.0) ma-user@bms-aiserver-pod05-97-200:/home/crq/ColossalAI/examples/language/llama/benchmark_7B/gemini_auto$ bash batch8_seq512.sh bash: UID: readonly variable Error: failed to...

qingccc

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

Courtesy-Xs

[Feature] optimize PP overlap

1

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

Edenzzzz

[Gemini] Prefetch next chunk before each op

1

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

botbw

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A...

isky-cd

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

flybird11111