stanford_alpaca icon indicating copy to clipboard operation
stanford_alpaca copied to clipboard

Fix custom token in train.py

Open naufalso opened this issue 2 years ago • 0 comments

After the LLaMA model finetuning using the existing training code, I realized that the model never outputs the EOS token, which causes the generation never stop until max_new_token is reached.

I tried to debug the code and found that tokenizer.eos_token, tokenizer.bos_token, and tokenizer.unk_token are all '' (empty string).

Since '' (empty string) is not equal to None, the custom tokens in the training code will not be added. So I would suggest fixing using the current code changes.

I have tested that after the training using the modified code, the model can output EOS token correctly.

naufalso avatar Apr 26 '23 01:04 naufalso