nebuly icon indicating copy to clipboard operation
nebuly copied to clipboard

[Chatllama] Should I add a <end_of_text> at the end of sentence?

Open bino282 opened this issue 1 year ago • 3 comments

When I train a actor model with bloom-560M. I realize that , the model generates the text repeated at the end. The model always generates enough words predefined by max_length but it;s not stop early. Should I add a <end_of_text> at the end of sentence when training?

bino282 avatar Mar 13 '23 04:03 bino282