save_config_to_hf_format()

[leejason]

For making "to_hf_weights.py" work correctly, do I have to modify the following if I have my own tokenizer trained with vocab_size=50400? Or, can I assume that "GPT2Tokenizer" does not matter here as long as I use my own tokenizer to decode the inference result? (I expected so but my experiments show negative results.)

def save_config_to_hf_format(......):

config = {
  ......
  "tokenizer_class": "GPT2Tokenizer"
  ......
}

Thank you for your kind advice.