RL4LMs icon indicating copy to clipboard operation
RL4LMs copied to clipboard

Bug while loading t5 base model

Open Sahajtomar opened this issue 1 year ago • 1 comments

I am trying to load t5 base model as per t5_ppo config. Strangely this error pops out. Works fine for t5-small.

	size mismatch for decoder.final_layer_norm.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([768]).
	size mismatch for lm_head.weight: copying a param with shape torch.Size([32128, 512]) from checkpoint, the shape in current model is torch.Size([32128, 768]).

Sahajtomar avatar Apr 25 '23 15:04 Sahajtomar