ColossalAI
ColossalAI copied to clipboard
[shardformer] fix pipeline forward error if custom layer distribution is used
β¦distribution
π Checklist before creating the PR
- [x] I have created an issue for this PR for traceability
- [x] The title follows the standard format:
[doc/gemini/tensor/...]: A concise description
- [ ] I have added relevant tags if possible for us to better distinguish different PRs
π¨ Issue number
Link this PR to your issue with words like fixed to automatically close the linked issue upon merge
e.g.
fixed #1234
,closed #1234
,resolved #1234
fixed #5187
π What does this PR do?
Summarize your work here. if you have any plots/diagrams/screenshots/tables, please attach them here.
Built-in model polices use self.distribute_layers
and self.get_stage_index
in every place so that pipeline forward works properly when a custom policy overrides those functions.
π₯ Checklist before requesting a review
- [x] I have linked my PR to an issue (instruction)
- [x] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
- [x] I have performed a self-review of my code
- [ ] I have added thorough tests.
- [ ] I have added docstrings for all the functions/methods I implemented
βοΈ Do you enjoy contributing to Colossal-AI?
- [ ] π Yes, I do.
- [ ] π No, I don't.
Tell us more if you don't enjoy contributing to Colossal-AI.
Thank you for your contribution.
Bot detected the issue body's language is not English, translate it automatically. π―ππ»π§βπ€βπ§π«π§πΏβπ€βπ§π»π©πΎβπ€βπ¨πΏπ¬πΏ
Thank you for your contribution.
It seems tests even didn't run due to timeout. Any hint why?