ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[DOC]: Fix a small place in README.md under ChatGPT demo folder

Open Fridge003 opened this issue 2 years ago • 8 comments

📚 The doc issue

In the "Train with real prompt data" part of ColossalAI/applications/ChatGPT/examples/README.md,

torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai

should be replaced by

torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai_gemini or torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai_zero2,

otherwise the program doesn't run.

Fridge003 avatar Feb 17 '23 04:02 Fridge003

是的,这里我也搞了半天,写清楚少走弯路

sxk000 avatar Feb 17 '23 07:02 sxk000

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Yes, I have spent a long time here too, writing clearly and avoiding detours

Issues-translate-bot avatar Feb 17 '23 07:02 Issues-translate-bot

image Train the reward model阶段你走通了吗? pretrain用哪个啊,在哪里下载啊?

sxk000 avatar Feb 17 '23 07:02 sxk000

@Fridge003 thanks for pointing this out. We will get to fix it up.

JThh avatar Feb 17 '23 13:02 JThh

image Train the reward model阶段你走通了吗? pretrain用哪个啊,在哪里下载啊?

pretrain不一定用下载的,只要是huggingface有的模型(比如'gpt2')都可以直接用。 这里我稍微修改了一下train_reward_model.py的源码,源码里面不知道为什么只用了bloom的模型,我给改成gpt2了,这样命令行的pretrain参数可以直接传'gpt2'. (但是'bloom'好像huggingface也有,所以你可以试试传一个'bloom')

稍微吐槽一下train_reward_model.py,model不知道为什么只有一个默认的bloom,如果可以的话最好像train_prompts.py那样改成三个, 然后传一个--model的参数。

Fridge003 avatar Feb 18 '23 06:02 Fridge003

To change pretrained models, kindly refer to issue #2781 for details.

JThh avatar Feb 18 '23 14:02 JThh

image Train the reward model阶段你走通了吗? pretrain用哪个啊,在哪里下载啊?

pretrain不一定用下载的,只要是huggingface有的模型(比如'gpt2')都可以直接用。 这里我稍微修改了一下train_reward_model.py的源码,源码里面不知道为什么只用了bloom的模型,我给改成gpt2了,这样命令行的pretrain参数可以直接传'gpt2'. (但是'bloom'好像huggingface也有,所以你可以试试传一个'bloom')

稍微吐槽一下train_reward_model.py,model不知道为什么只有一个默认的bloom,如果可以的话最好像train_prompts.py那样改成三个, 然后传一个--model的参数。

按照@JThh 提示,不需要任何更改,这样运行就可以了:python train_reward_model.py --pretrain bigscience/bloom-560m 非常感谢各位大佬!

sxk000 avatar Feb 20 '23 03:02 sxk000

To change pretrained models, kindly refer to issue #2781 for details.

it works ! thanks very much !

sxk000 avatar Feb 20 '23 03:02 sxk000