gpt-neo-fine-tuning-example

gpt-neo-fine-tuning-example copied to clipboard

If I want to save and run generation on the model later on, I assume I do something like this:

After training: tokenizer.save_pretrained('./results/')

Later generation:

weights = "./results/"
tokenizer = GPT2Tokenizer.from_pretrained(weights, bos_token='<|startoftext|>', eos_token='<|endoftext|>', pad_token='<|pad|>')
model = GPTNeoForCausalLM.from_pretrained('./results/checkpoint-90/').cuda()

But I get an error about the size not being right. Any idea why?

File "generate.py", line 12, in <module>
    model = GPTNeoForCausalLM.from_pretrained('./results/checkpoint-90').cuda()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py", line 1183, in from_pretrained
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for GPTNeoForCausalLM:
	size mismatch for transformer.wte.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([50258, 768]).