Baichuan-13B v100 32g 单卡模型加载完成，对话报错RuntimeError: "addmm_impl_cpu

v100 32g 单卡模型加载完成，对话报错RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

Open huangb5760 opened this issue 1 year ago • 5 comments

File "/home/ubuntu/anaconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/Baichuan-13B-Chat/modeling_baichuan.py", line 370, in forward layer_outputs = decoder_layer( File "/home/ubuntu/anaconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/Baichuan-13B-Chat/modeling_baichuan.py", line 193, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/home/ubuntu/anaconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/Baichuan-13B-Chat/modeling_baichuan.py", line 124, in forward proj = self.W_pack(hidden_states) File "/home/ubuntu/anaconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/anaconda3/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: "addmm_impl_cpu" not implemented for 'Half'