[Feature] 'Add_special_tokens' parameter is needed

[SefaZeng]

Describe the feature

As the base models (pre-trained models without sft) do not predict after the EOS token during the pre-train stage, setting the add_special_tokens=True by default is not the right way to evaluate these models' performance. So adding a parameter for this would be good?

Will you implement it?

• [ ] I would like to implement this feature and create a PR!

[yingfhu]

Good one, but we did not conduct each experiment to compare the differences, it is better that you have some evaluation results that substantiate your issue. BTW, you are welcomed to implement this feature to support add_special_tokens option in model.

[bittersweet1999]

Feel free to reopen it if needed