ChatGLM-Finetuning
ChatGLM-Finetuning copied to clipboard
Is this line wrong?
https://github.com/liucongg/ChatGLM-Finetuning/blob/d52202a2facfa1dff45b0daec7e56aa54c126616/utils.py#L95-L96
I am not very sure. But this seems the right code:
input_ids = tokenizer.convert_tokens_to_ids(tokens) + [tokenizer.get_command("[gMASK]"), tokenizer.get_command("sop")]