text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

One-click installer does not install requirements for 8bit support

Open MrCheeze opened this issue 1 year ago • 0 comments

Describe the bug

It would be nice for this to work out of the box, especially since the one-click installers are meant to be "for dummies".

Is there an existing issue for this?

  • [X] I have searched the existing issues

Reproduction

python server.py --auto-devices --load-in-8bit

Screenshot

No response

Logs

Traceback (most recent call last):
File “C:\Users\Matthew\Documents\oobabooga-windows\text-generation-webui\server.py”, line 100, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name)
File “C:\Users\Matthew\Documents\oobabooga-windows\text-generation-webui\modules\models.py”, line 186, in load_model
model = LoaderClass.from_pretrained(checkpoint, **params)
File “C:\Users\Matthew\Documents\oobabooga-windows\installer_files\env\lib\site-packages\transformers\models\auto\auto_factory.py”, line 471, in from_pretrained
return model_class.from_pretrained(
File “C:\Users\Matthew\Documents\oobabooga-windows\installer_files\env\lib\site-packages\transformers\modeling_utils.py”, line 2231, in from_pretrained
raise ImportError(
ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate and the latest version of bitsandbytes pip install -i https://test.pypi.org/simple/ bitsandbytes or pip install bitsandbytes`

System Info

Windows 11, GTX 1660

MrCheeze avatar Apr 19 '23 17:04 MrCheeze