RL4LMs
RL4LMs copied to clipboard
Bug while loading t5 base model
I am trying to load t5 base model as per t5_ppo config. Strangely this error pops out. Works fine for t5-small.
size mismatch for decoder.final_layer_norm.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([768]).
size mismatch for lm_head.weight: copying a param with shape torch.Size([32128, 512]) from checkpoint, the shape in current model is torch.Size([32128, 768]).