Hongxin Liu
Hongxin Liu
## Describe the problem In version 0.1.7, I found that Op hook leads to memory leak. If you use the hook on nn.module, even though it's a dummy hook, more...
### Describe the feature We set spec on parameter now, which means each paramter has its own unchanged compute_pattern. However, some models, like GPT-2, share parameter among different layers. GPT-2...
### 📚 The doc issue Mentioned in #709 . Colossal requires torch>=1.8, minimal version of CUDA for this torch version is cu111. Maybe we can add a simple Colab sample...
## What does this PR do? Fixes https://github.com/hpcaitech/ColossalAI/issues/1330 Fixes https://github.com/Lightning-AI/lightning/issues/12733 Add ColossalAI strategy which supports ZeRO-DP with chunk-based memory management. ### Does your PR introduce any breaking changes? If yes,...
## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...
### 📚 The doc issue A new user is hard to start training after reading the readme. I list some FAQs which users may concern: - How to prepare training...
## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...
## 📌 Checklist before creating the PR - [x] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A concise...
### Proposal ## Motivation 1. Current initialization process is difficult and hard to maintain. It contains hundreds of hard code `if-else`, which is hard to read and modify. 2. Current...