ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[autoparallel] apply repeat block to reduce solving time

Open YuliangLiu0306 opened this issue 2 years ago • 1 comments

📌 Checklist before creating the PR

  • [x] I have created an issue for this PR for traceability
  • [x] The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • [x] I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

  • Build alias set in StrategyConstructor for all repeating blocks.
  • Remove graph analysis from auto parallel workflow
  • Using alias set to prune the solving space

💥 Checklist before requesting a review

  • [x] I have linked my PR to an issue (instruction)
  • [x] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • [x] I have performed a self-review of my code
  • [x] I have added thorough tests.
  • [x] I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • [x] 🌝 Yes, I do.
  • [ ] 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

YuliangLiu0306 avatar Feb 27 '23 03:02 YuliangLiu0306

The code coverage for the changed files is 16%.

Click me to view the complete report
Name                                                                                 Stmts   Miss  Cover
--------------------------------------------------------------------------------------------------------
colossalai/auto_parallel/tensor_shard/initialize.py                                    118     91    23%
colossalai/auto_parallel/tensor_shard/solver/solver.py                                 275    252     8%
colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py                 116     97    16%
tests/test_auto_parallel/test_tensor_shard/test_gpt/test_solver_with_gpt_module.py      65     45    31%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/utils.py                  122    105    14%
tests/test_auto_parallel/test_tensor_shard/test_solver_with_resnet_v2.py                54     41    24%
--------------------------------------------------------------------------------------------------------
TOTAL                                                                                  750    631    16%

github-actions[bot] avatar Feb 27 '23 04:02 github-actions[bot]