gpt-fast icon indicating copy to clipboard operation
gpt-fast copied to clipboard

KeyError: 'model.layers.{}.self_attn.W_pack.weight'

Open wccccp opened this issue 1 year ago • 4 comments

device: torch.device = torch.device(torch._C._get_default_device()), # torch.device('cpu'), Model config {'block_size': 2048, 'vocab_size': 32000, 'n_layer': 32, 'n_head': 32, 'dim': 4096, 'intermediate_size': 11008, 'n_local_heads': 32, 'head_dim': 128, 'rope_base': 10000, 'norm_eps': 1e-05} /mnt/user/wangchenpeng/venv/fast/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() Traceback (most recent call last): File "scripts/convert_hf_checkpoint.py", line 106, in convert_hf_checkpoint( File "/mnt/user/wangchenpeng/venv/fast/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "scripts/convert_hf_checkpoint.py", line 76, in convert_hf_checkpoint new_key = weight_map[abstract_key] KeyError: 'model.layers.{}.self_attn.W_pack.weight'

wccccp avatar Dec 05 '23 07:12 wccccp

What command are you running to get this error?

Chillee avatar Dec 05 '23 21:12 Chillee

What command are you running to get this error?

I loaded hugging face model inference locally and this happened.

wccccp avatar Dec 08 '23 14:12 wccccp

你是不是拉的是baichuan的模型,因为baichuan模型虽然和llama属于是双胞胎,但是他们在模型文件中也是有区别的,在做映射那一步的时候底层写的只是针对llama的,所以你会报这个错误,解决办法就是换llama

Joker-sad avatar Dec 12 '23 06:12 Joker-sad

你是不是拉的是baichuan的模型,因为baichuan模型虽然和llama属于是双胞胎,但是他们在模型文件中也是有区别的,在做映射那一步的时候底层写的只是针对llama的,所以你会报这个错误,解决办法就是换llama

现在只能支持llama的吗😄

wccccp avatar Jan 05 '24 05:01 wccccp