OpenChatKit icon indicating copy to clipboard operation
OpenChatKit copied to clipboard

[feature]Do you support RLHF training ?

Open ht-zhou opened this issue 1 year ago • 3 comments

After viewing your code , I found that you haven't support RLHF training yet. Your code is mainly about distributed training using pipeline & data parallel. Do you have the plan to support RLHF training?Do you think it is necessary?

ht-zhou avatar Mar 14 '23 01:03 ht-zhou

Gostei

VANESSINHAS2023 avatar Mar 14 '23 04:03 VANESSINHAS2023

@zhangce, what do you think?

csris avatar Mar 18 '23 05:03 csris

Impressive to meet an opensource version of the chatGPT. I cannot imagine how tremendous the service to be done for releasing this. Nevertheless, IMHO, current performance based on my experience... I dunno the exact cause but much to be done left I guess. I suspect one of them is RLHF if the quality/distribution of the instruction dataset and the pretrained language model (Neo-20B) is trusted.

sonsus avatar Mar 21 '23 02:03 sonsus