text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Published 20 hours ago • verified publisher icon oobabooga

Absoultely broken

Open nextdimension opened this issue 1 year ago • 0 comments

The requirements.txt use a version of bitsandbytes that is not compiled for GPU support, so this fails to work on linux.

I have installed version 0.37 of b&b from pip and it is not compiled with gpu support. I tried every version of b&b and combination of cuda versions and I either get not compiled with gpu support error or AttributeError: module 'bitsandbytes.nn' has no attribute 'Linear8bitLt'

I can also find no version specifically compiled for cuda 12.0 or 11.8 but they are mentioned as having better cuda detection in the 0.37 release notes.

The readme is broken, it does not work with GPU support or on Linux

nextdimension avatar Mar 08 '23 18:03 nextdimension