ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

why GPTActor is not to load pretained parameters in chatgpt examples

Open alphanlp opened this issue 2 years ago • 2 comments

🐛 Describe the bug

with strategy.model_init_context(): if args.model == 'gpt2': actor = GPTActor().cuda() critic = GPTCritic().cuda()

Environment

No response

alphanlp avatar Feb 24 '23 03:02 alphanlp

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Title: why GPTActor is not to load pretrained parameters in chatgpt examples

Issues-translate-bot avatar Feb 24 '23 03:02 Issues-translate-bot

As it just shows training process, we just use randomly initialized model for simplicity. You can use a pretrained model easily by GPTActor(pretrained='gpt2').

ver217 avatar Feb 28 '23 07:02 ver217

https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat We have updated a lot. This issue was closed due to inactivity. Thanks.

binmakeswell avatar Apr 20 '23 10:04 binmakeswell