运行generate.sh报错

[jcyao]

运行generate.sh加载模型正常，gradio发送消息报错，请问哪里出了问题吗？bitsandbytes有bug？ File "/home/yjc/LaWGPT/src/generate.py", line 131, in generate_with_callback model.generate(**kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/peft/peft_model.py", line 580, in generate return self.base_model.generate(**kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/generation/utils.py", line 1604, in generate return self.beam_search( File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/generation/utils.py", line 2902, in beam_search outputs = self( File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 688, in forward outputs = self.model( File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 578, in forward layer_outputs = decoder_layer( File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 293, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 197, in forward query_states = self.q_proj(hidden_states).view(bsz, q_len, self.num_heads, self.head_dim).transpose(1, 2) File "/home/yjc/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/yjc/anaconda3/lib/python3.9/site-packages/peft/tuners/lora.py", line 502, in forward result = super().forward(x) File "/home/yjc/anaconda3/lib/python3.9/site-packages/bitsandbytes/nn/modules.py", line 320, in forward out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state) File "/home/yjc/anaconda3/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 500, in matmul return MatMul8bitLt.apply(A, B, out, bias, state) File "/home/yjc/anaconda3/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 380, in forward outliers = state.CB[:, state.idx.long()].clone() TypeError: 'NoneType' object is not subscriptable