RWKV-LM icon indicating copy to clipboard operation
RWKV-LM copied to clipboard

rwkv-7 代码和模型不一致

Open qxde01 opened this issue 9 months ago • 1 comments

rwkv_v7_demo.py : args.vocab_size = 50304 01.b 实际:65536

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for RWKV: Missing key(s) in state_dict: "blocks.0.att.v0", "blocks.0.att.v1", "blocks.0.att.v2". size mismatch for emb.weight: copying a param with shape torch.Size([65536, 768]) from checkpoint, the shape in current model is torch.Size([50304, 768]). size mismatch for head.weight: copying a param with shape torch.Size([65536, 768]) from checkpoint, the shape in current model is torch.Size([50304, 768]).

qxde01 avatar Jan 26 '25 04:01 qxde01