ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

[BUG]: train_gpt_demo.py cannot load parameters from model file

Open zhanghaoie opened this issue 2 years ago • 1 comments

🐛 Describe the bug

I want to load model parameters from a pretrained model after use tp, but use train_gpt_demo.py after zero_model_wrapper also made tensor shape mismatch error

Environment

No response

zhanghaoie avatar Feb 22 '23 11:02 zhanghaoie

Hi @zhanghaoie maybe you can refer to this example https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT#how-to-saveload-checkpoint

binmakeswell avatar Feb 28 '23 08:02 binmakeswell

We have updated a lot. This issue was closed due to inactivity. Thanks.

binmakeswell avatar Apr 20 '23 07:04 binmakeswell