Mismatch Between Model and Checkpoint When Loading MiniCPM-V-2_6-int4

[yeganebagheri]

Hello,

When I run the code, I encounter an issue indicating a mismatch between the model and the checkpoint. I cloned the MiniCPM-V-2_6-int4 model from Hugging Face as mentioned in the README, but it seems to be incompatible with the checkpoint.

Here is the error message I received:

RuntimeError: Error in segment_caption:
 Error(s) in loading state_dict for MiniCPMV:
        size mismatch for llm.model.layers.0.self_attn.q_proj.weight: copying a param with shape torch.Size([6422528, 1]) from checkpoint, the shape in current model is torch.Size([3584, 3584]).
        size mismatch for llm.model.layers.0.self_attn.k_proj.weight: copying a param with shape torch.Size([917504, 1]) from checkpoint, the shape in current model is torch.Size([512, 3584]).
        size mismatch for llm.model.layers.0.self_attn.v_proj.weight: copying a param with shape torch.Size([917504, 1]) from checkpoint, the shape in current model is torch.Size([512, 3584]).
        size mismatch for llm.model.layers.0.self_attn.o_proj.weight: copying a param with shape torch.Size([6422528, 1]) from checkpoint, the shape in current model is torch.Size([3584, 3584]).
        size mismatch for llm.model.layers.0.mlp.gate_proj.weight: copying a param with shape torch.Size([33947648, 1]) from checkpoint, the shape in current model is torch.Size([18944, 3584]).

I would appreciate any help in resolving this issue. Thank you!