duanjunwen

[feat] add DistributedAdafactor;

1

## 📌 Checklist before creating the PR - [x] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A concise...

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

[Fix] Update MoeHybridParallelPlugin;

1

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

[QUESTION] Question about ZeroBubbleVPipeScheduler schedule W

1

Hello, I had some questions while reading ZeroBubbleVPipeScheduler, especially schedule W. (As the title) Can you plz explain exactly how this part is implemented ? In detailed: 1) Where it...

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [x] The title follows the standard format: `[doc/gemini/tensor/...]: A...

### Describe the feature This issue contains requirements for the zerobubble pp. Source Github Link: https://github.com/sail-sg/zero-bubble-pipeline-parallelism Source Paper: https://arxiv.org/abs/2401.10241

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...