ColossalAI [BUG]: python pretrain.py /home/test/anaconda3/envs/colossalai_0.3.6/lib/python3.8/site-packages/colossalai/shardformer/layer/normalization.py:45: UserWarning: Please install apex from source (https://github.com/NVIDIA/apex) to use the fused layernorm kernel warnings.warn("Please install apex from source (https://github.com/NVIDIA/apex) to use the fused layernorm kernel") /home/test/anaconda3/envs/colossalai_0.3.6/lib/python3.8/site-packages/colossalai/initialize.py:48: UserWarning: `config` is deprecated and will be removed soon. warnings.warn("`config` is deprecated and will be removed soon.")

[BUG]: python pretrain.py /home/test/anaconda3/envs/colossalai_0.3.6/lib/python3.8/site-packages/colossalai/shardformer/layer/normalization.py:45: UserWarning: Please install apex from source (https://github.com/NVIDIA/apex) to use the fused layernorm kernel warnings.warn("Please install apex from source (https://github.com/NVIDIA/apex) to use the fused layernorm kernel") /home/test/anaconda3/envs/colossalai_0.3.6/lib/python3.8/site-packages/colossalai/initialize.py:48: UserWarning: `config` is deprecated and will be removed soon. warnings.warn("`config` is deprecated and will be removed soon.")

Open liqwertyu opened this issue 1 year ago • 0 comments

运行示例llama2的pretrain.py时，会出现这样类似卡主的情况

CUDA Version: V11.1.105 Python Version: Python 3.8.18 PyTorch Version: 2.0.0+cu117

Mar 13 '24 05:03 liqwertyu