ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

Can it support t5 or mt5 model training?

Open RyanYip-Kat opened this issue 3 years ago • 2 comments

Describe the feature

If I want to train t5 or mt5 model with my dataset,whether it can be implemented using Colossal AI ?

RyanYip-Kat avatar Dec 02 '22 05:12 RyanYip-Kat

https://github.com/hpcaitech/ColossalAI/blob/main/examples/language/gpt/README.md Of course, ColossalAI is a general framework. You can refer the GPT example.

feifeibear avatar Dec 02 '22 06:12 feifeibear

I am currently trying to train a mt5-large model using colossalai but seems to have some issue with setstorage bug #2145. I would be happy to contribute a working example if this bug was sorted out.

theblackcat102 avatar Dec 18 '22 10:12 theblackcat102

Welcome contributions from the community. https://github.com/hpcaitech/ColossalAI/tree/main/examples/community#looking-for-examples This issue was closed due to inactivity. Thanks.

binmakeswell avatar Apr 14 '23 08:04 binmakeswell