ColossalAI

ColossalAI copied to clipboard

### Describe the feature Please create wheel for windows. other tools like deep speed, detectron2, pytorch3d, triton have started to provide wheel for Windows users. Because of this tool we...

nitinmukesh

[BUG]: ModuleNotFoundError: No module named 'colossalai._C.cpu_adam'

5

### 🐛 Describe the bug **I got some errors when running resnet.** `(colossal-AI) [root@node64 resnet]# colossalai run --nproc_per_node 1 train.py -c ./ckpt-fp32 [07/25/23 20:27:25] INFO colossalai - colossalai - INFO:...

Aaricis

能否提供一个可以直接运行的grpo数据集

1

想要运行grpo，根据readme开始执行，下载[qwedsacf/competition_math dataset](https://huggingface.co/datasets/qwedsacf/competition_math)数据集 prepare_prompt_dataset.sh开始就出现错误，然后就运行不下去了。 希望作者可以对readme的可复现方式详细说明一下，谢谢

tonylin52

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

YeAnbang

[BUG]: multi-node training stuck for open-sora

1

### Is there an existing issue for this bug? - [x] I have searched the existing issues ### The bug has not been fixed in the latest main branch -...

ltm920716

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

xysheng-colossal

[DOC]: 如何在低版本的CUDA上运行Colossal-AI

1

### 📚 The doc issue 目前CUDA版本是11.4，限制了pytorch只能安装1.12，但是官方列举的要求除了torch2.2，其它条件都满足。请问有什么办法可以使用Colossal-AI去微调模型吗？

wenzyan

[checkpoint_io] Fix gather_state_dict_fast

1

## 🚨 Issue number fixed #6248 ## 📝 What does this PR do? This PR fixed 2 issues: * For `reqs = dist.batch_isend_irecv(ops)` it's not guaranteed that len(ops) == len(reqs)...

pbelevich

[Ring Attention] Add more detailed references

1

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

Edenzzzz

[BUG]: RuntimeError: Failed to replace input_layernorm of type DeepseekV3RMSNorm with FusedRMSNorm with the exception: 'NoneType' object is not callable.

2

### Is there an existing issue for this bug? - [x] I have searched the existing issues ### The bug has not been fixed in the latest main branch -...

lgybuaa