[BUG]: inference.py error

[qq31682216]

🐛 Describe the bug

按照example的说明跑了一下，发现inference会报格式错误。 命令：python inference.py --model_path ./actor_checkpoint_prompts.pt --pretrain bigscience/bloom-560m --model bloom 输出： Traceback (most recent call last): File "inference.py", line 59, in eval(args) File "inference.py", line 23, in eval actor.model.load_state_dict(state_dict) File "/home/sre/anaconda3/envs/gpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for BloomForCausalLM: Missing key(s) in state_dict: "transformer.word_embeddings.weight", "transformer.word_embeddings_layernorm.weight", "transformer.word_embeddings_layernorm.bias" ...

Environment

nvcc -V Cuda compilation tools, release 11.7, V11.7.99 Build cuda_11.7.r11.7/compiler.31442593_0

[qq31682216]

example：https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT/examples

[JThh]

Try adding strict=False to this line.

[qq31682216]

引用的那一行是无法添加strict=False这个参数的，我看了源码确实不支持。 23行我改为actor.model.load_state_dict(state_dict, strict=False)依然报错： Traceback (most recent call last): File "inference.py", line 60, in eval(args) File "inference.py", line 23, in eval actor.model.load_state_dict(state_dict, strict=False) File "/home/sre/anaconda3/envs/gpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for BloomForCausalLM: size mismatch for transformer.ln_f.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]). size mismatch for transformer.ln_f.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]). size mismatch for lm_head.weight: copying a param with shape torch.Size([50257, 768]) from checkpoint, the shape in current model is torch.Size([250880, 1024]).

[JThh]

I guess we can merge the issue with #3061 and request @ht-zhou's help on this.