[BUG]: ImportError: cannot import name 'MOE_CONTEXT' from partially initialized module

[songty21110133]

🐛 Describe the bug

您好，当我只调用from colossalai.nn.layer.moe import Experts, MoeLayer, Top1Router, NormalNoiseGenerator时候出现以下问题： Traceback (most recent call last): File "/home/songty/working_dir/moe/main.py", line 2, in <module> from colossalai.nn.layer.moe import Experts, MoeLayer, Top1Router, NormalNoiseGenerator File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/__init__.py", line 1, in <module> from .checkpoint import load_moe_model, save_moe_model File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/checkpoint.py", line 5, in <module> from .experts import MoeExperts File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/experts.py", line 9, in <module> from colossalai.context.moe_context import MOE_CONTEXT File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/context/moe_context.py", line 7, in <module> from colossalai.legacy.tensor import ProcessGroup File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/legacy/__init__.py", line 1, in <module> from .initialize import ( File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/legacy/initialize.py", line 19, in <module> from colossalai.context.moe_context import MOE_CONTEXT ImportError: cannot import name 'MOE_CONTEXT' from partially initialized module 'colossalai.context.moe_context' (most likely due to a circular import)

Environment

Python 3.10.6 Pytorch 2.0.1 CUDA 12.0 ColossalAI 0.3.3

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

🐛 Describe the bug

Hello, when I just call from colossalai.nn.layer.moe import Experts, MoeLayer, Top1Router, NormalNoiseGenerator, the following problem occurs: Traceback (most recent call last): File "/home/songty/working_dir/moe/main.py", line 2, in <module> from colossalai.nn.layer.moe import Experts, MoeLayer, Top1Router, NormalNoiseGenerator File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/__init__.py", line 1, in <module> from .checkpoint import load_moe_model, save_moe_model File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/checkpoint.py", line 5, in <module> from .experts import MoeExperts File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/nn/layer/moe/experts.py", line 9, in <module> from colossalai.context.moe_context import MOE_CONTEXT File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/context/moe_context.py", line 7, in <module> from colossalai.legacy.tensor import ProcessGroup File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/legacy/__init__.py", line 1, in <module> from .initialize import ( File "/home/songty/miniconda3/envs/songty_pytorch/lib/python3.10/site-packages/colossalai/legacy/initialize.py", line 19, in <module> from colossalai.context.moe_context import MOE_CONTEXT ImportError: cannot import name 'MOE_CONTEXT' from partially initialized module 'colossalai.context.moe_context' (most likely due to a circular import)

Environment

Python 3.10.6 Pytorch 2.0.1 CUDA 12.0 ColossalAI 0.3.3

[flybird11111]

Hi, 'colossalai.nn.layer.moe' will be deprecated soon. you can follow the open-moe, https://github.com/hpcaitech/ColossalAI/blob/main/applications/ColossalMoE/train.py