ColossalAI
ColossalAI copied to clipboard
[autoparallel] apply repeat block to reduce solving time
📌 Checklist before creating the PR
- [x] I have created an issue for this PR for traceability
- [x] The title follows the standard format:
[doc/gemini/tensor/...]: A concise description - [x] I have added relevant tags if possible for us to better distinguish different PRs
🚨 Issue number
Link this PR to your issue with words like fixed to automatically close the linked issue upon merge
e.g.
fixed #1234,closed #1234,resolved #1234
📝 What does this PR do?
- Build alias set in StrategyConstructor for all repeating blocks.
- Remove graph analysis from auto parallel workflow
- Using alias set to prune the solving space
💥 Checklist before requesting a review
- [x] I have linked my PR to an issue (instruction)
- [x] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
- [x] I have performed a self-review of my code
- [x] I have added thorough tests.
- [x] I have added docstrings for all the functions/methods I implemented
⭐️ Do you enjoy contributing to Colossal-AI?
- [x] 🌝 Yes, I do.
- [ ] 🌚 No, I don't.
Tell us more if you don't enjoy contributing to Colossal-AI.
The code coverage for the changed files is 16%.
Click me to view the complete report
Name Stmts Miss Cover
--------------------------------------------------------------------------------------------------------
colossalai/auto_parallel/tensor_shard/initialize.py 118 91 23%
colossalai/auto_parallel/tensor_shard/solver/solver.py 275 252 8%
colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py 116 97 16%
tests/test_auto_parallel/test_tensor_shard/test_gpt/test_solver_with_gpt_module.py 65 45 31%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/utils.py 122 105 14%
tests/test_auto_parallel/test_tensor_shard/test_solver_with_resnet_v2.py 54 41 24%
--------------------------------------------------------------------------------------------------------
TOTAL 750 631 16%