How to add a large amount of new special tokens?

[Zhang-Each]

Reminder

• [x] I have read the above rules and searched the existing issues.

System Info

I plan to add a large amount of special tokens to the LLM vocab (e.g. 1000 new tokens). I found that the parameter--new_special_tokens can be used but what if I want to add more tokens? Should I pass all the new tokens in the parameter or there is a more flexible way? Thank you!

Reproduction

Put your message here.

Others

No response

[Kuangdd01]

You can refer to this PR https://github.com/hiyouga/LLaMA-Factory/pull/9267