'Parameter' object has no attribute 'CB' 使用int8 启动时报错

[LymanLiuChina]

File "/opt/conda/lib/python3.8/site-packages/tensor_parallel/wrapper.py", line 71, in forward output = self.tp_wrapped_module(*args, **kwargs) File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/opt/conda/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 313, in forward if self.weight.CB is not None: AttributeError: 'Parameter' object has no attribute 'CB'

[fengyh3]

你用的是哪个模型呢？能不能提供更多的信息？

[fengyh3]

@LymanLiuChina 已修复。https://github.com/fengyh3/llama_inference/tree/main