How did you expand vocabulary before training the model?

[mohataher]

Hi there @lianjiatech, great work.

I'm trying to reproduce your work on the vocabulary expansion. But I couldn't find the code responsible for this step.

I have only found this but it's commented out and I'm not sure if it's the right spot.

Would you please help me find the code that performed tokenizer re-training and vocabulary expansion of the model please?