ChatGLM-Finetuning icon indicating copy to clipboard operation
ChatGLM-Finetuning copied to clipboard

Is this line wrong?

Open iridescentee opened this issue 10 months ago • 0 comments

https://github.com/liucongg/ChatGLM-Finetuning/blob/d52202a2facfa1dff45b0daec7e56aa54c126616/utils.py#L95-L96

I am not very sure. But this seems the right code:

input_ids = tokenizer.convert_tokens_to_ids(tokens) + [tokenizer.get_command("[gMASK]"), tokenizer.get_command("sop")]

iridescentee avatar Aug 17 '23 09:08 iridescentee