text-generation-webui
text-generation-webui copied to clipboard
One-click installer does not install requirements for 8bit support
Describe the bug
It would be nice for this to work out of the box, especially since the one-click installers are meant to be "for dummies".
Is there an existing issue for this?
- [X] I have searched the existing issues
Reproduction
python server.py --auto-devices --load-in-8bit
Screenshot
No response
Logs
Traceback (most recent call last):
File “C:\Users\Matthew\Documents\oobabooga-windows\text-generation-webui\server.py”, line 100, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name)
File “C:\Users\Matthew\Documents\oobabooga-windows\text-generation-webui\modules\models.py”, line 186, in load_model
model = LoaderClass.from_pretrained(checkpoint, **params)
File “C:\Users\Matthew\Documents\oobabooga-windows\installer_files\env\lib\site-packages\transformers\models\auto\auto_factory.py”, line 471, in from_pretrained
return model_class.from_pretrained(
File “C:\Users\Matthew\Documents\oobabooga-windows\installer_files\env\lib\site-packages\transformers\modeling_utils.py”, line 2231, in from_pretrained
raise ImportError(
ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate and the latest version of bitsandbytes pip install -i https://test.pypi.org/simple/ bitsandbytes or pip install bitsandbytes`
System Info
Windows 11, GTX 1660