ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[shardformer] fix pipeline forward error if custom layer distribution is used

Open insujang opened this issue 1 year ago β€’ 3 comments

…distribution

πŸ“Œ Checklist before creating the PR

  • [x] I have created an issue for this PR for traceability
  • [x] The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • [ ] I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

fixed #5187

πŸ“ What does this PR do?

Summarize your work here. if you have any plots/diagrams/screenshots/tables, please attach them here.

Built-in model polices use self.distribute_layers and self.get_stage_index in every place so that pipeline forward works properly when a custom policy overrides those functions.

πŸ’₯ Checklist before requesting a review

  • [x] I have linked my PR to an issue (instruction)
  • [x] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • [x] I have performed a self-review of my code
  • [ ] I have added thorough tests.
  • [ ] I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • [ ] 🌝 Yes, I do.
  • [ ] 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

insujang avatar Dec 15 '23 04:12 insujang

Thank you for your contribution.

flybird11111 avatar Dec 16 '23 08:12 flybird11111

Bot detected the issue body's language is not English, translate it automatically. πŸ‘―πŸ‘­πŸ»πŸ§‘β€πŸ€β€πŸ§‘πŸ‘«πŸ§‘πŸΏβ€πŸ€β€πŸ§‘πŸ»πŸ‘©πŸΎβ€πŸ€β€πŸ‘¨πŸΏπŸ‘¬πŸΏ


Thank you for your contribution.

Issues-translate-bot avatar Dec 16 '23 08:12 Issues-translate-bot

It seems tests even didn't run due to timeout. Any hint why?

insujang avatar Jan 23 '24 05:01 insujang