ize mismatch for embedding.weight: copying a param with shape torch.Size([128, 229376]) from checkpoint, the shape in current model is torch.Size([128, 4096]).

[CyanMystery]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

Traceback (most recent call last): File "F:\ChatGLM-6B\ptuning\main.py", line 434, in main() File "F:\ChatGLM-6B\ptuning\main.py", line 125, in main model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict) File "E:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for PrefixEncoder: Missing key(s) in state_dict: "trans.0.weight", "trans.0.bias", "trans.2.weight", "trans.2.bias". size mismatch for embedding.weight: copying a param with shape torch.Size([128, 229376]) from checkpoint, the shape in current model is torch.Size([128, 4096]).

Expected Behavior

我从ptuning checkpoint 进行训练 并且resume form checkpoint

Steps To Reproduce

tionConfig { "_from_model_config": true, "bos_token_id": 130004, "eos_token_id": 130005, "pad_token_id": 3, "transformers_version": "4.27.1", "use_cache": false }

ChatGLMConfig { "_name_or_path": "F:\ChatGLM-6B\THUDM\chatglm-6b", "architectures": [ "ChatGLMModel" ], "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration" }, "bos_token_id": 130004, "eos_token_id": 130005, "gmask_token_id": 130001, "hidden_size": 4096, "inner_hidden_size": 16384, "layernorm_epsilon": 1e-05, "mask_token_id": 130000, "max_sequence_length": 2048, "model_type": "chatglm", "num_attention_heads": 32, "num_layers": 28, "pad_token_id": 3, "position_encoding_2d": true, "pre_seq_len": 128, "prefix_projection": true, "quantization_bit": 0, "torch_dtype": "float16", "transformers_version": "4.27.1", "use_cache": false, "vocab_size": 130528 }

Environment

windows 10
python 3.9
transformer 4.27.1
torch 2.0.1 cuda

Anything else?

No response