ImportError when trying to use 'Linear8bitLt' from 'lit_llama.quantization'

[splendidcomputer]

I'm encountering an error while trying to use bitsandbytes in lit-llama. Here's the error message:

Loading model ...
bin /home/mostafa/anaconda3/envs/ll/lib/python3.11/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
Traceback (most recent call last):
File "/home/mostafa/programming/lit-llama/generate.py", line 170, in <module>
CLI(main)
File "/home/mostafa/anaconda3/envs/ll/lib/python3.11/site-packages/jsonargparse/_cli.py", line 85, in CLI
return _run_component(component, cfg_init)
File "/home/mostafa/anaconda3/envs/ll/lib/python3.11/site-packages/jsonargparse/_cli.py", line 147, in _run_component
return component(**cfg)
File "/home/mostafa/programming/lit-llama/generate.py", line 129, in main
with fabric.init_module(empty_init=True), quantization(mode=quantize):
File "/home/mostafa/anaconda3/envs/ll/lib/python3.11/contextlib.py", line 137, in enter
return next(self.gen)
File "/home/mostafa/programming/lit-llama/lit_llama/utils.py", line 142, in quantization
from .quantization import Linear8bitLt
ImportError: cannot import name 'Linear8bitLt' from 'lit_llama.quantization' (/home/mostafa/programming/lit-llama/lit_llama/quantization.py)

I receive this error when I run the following command:

python generate.py --quantize llm.int8 --prompt "Hello, my name is"

I'm using Python 3.11, and the error seems to be with importing 'Linear8bitLt' from 'lit_llama.quantization'. Could you please help me with this issue?

Thank you.

[carmocca]

It might be caused by https://github.com/TimDettmers/bitsandbytes/issues/544: pip install scipy should fix it in that case

[splendidcomputer]

It might be caused by TimDettmers/bitsandbytes#544: pip install scipy should fix it in that case

It works now. Thank you so much for your great help.