RuntimeError: Error(s) in loading state_dict for MPLUGOwl2LlamaForCausalLM: size mismatch for norm.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([4096]). You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.

[style09131111]

RuntimeError: Error(s) in loading state_dict for MPLUGOwl2LlamaForCausalLM: size mismatch for norm.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([4096]). How can i solve this question,thank you!

[CarlCloudWang]

你试试直接把模型拆开来自己写个__init__加载模型，从scorer.py看，实际上这模型就三个部分： tokenizer = AutoTokenizer.from_pretrained(model_path, model_path=model_path, use_fast=False, cache_dir=cache_dir) model = MPLUGOwl2LlamaForCausalLM.from_pretrained(model_path, model_path=model_path, local_files_only=True, cache_dir=cache_dir, low_cpu_mem_usage=True, device_map="auto") image_processor = CLIPImageProcessor.from_pretrained(model_path) 在Modeling_llama.py的结尾，作者很巧妙地直接把llama2的函数动态加载了，所以随便加载权重。