text-generation-webui
text-generation-webui copied to clipboard
oobabooga
Too many problems installing on Debian 12
Describe the bug
start_linux.sh fails
The following packages were not installed with ./start_linux.sh and had to be installed manually into the ${HOME}/src/text-generation-webui/installer_files/env conda environment.
#conda install gradio #conda install pyyaml #conda install accelerate #conda install pillow #conda install markdown #conda install transformers #conda install numba #conda install datasets #conda install peft
Running server.py fails on exception (see logs included)
It would be much cleaner and easier if the installation were not a 1-click-fits-all, but rather
- instruction to create the correct conda environment
- instruction on installing and configuring Oobabooga
- instructions on starting server
Is there an existing issue for this?
- [X] I have searched the existing issues
Reproduction
On Debian 12, follow instructions from README.md
Screenshot
No response
Logs
when running "python ./server.py"
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/__init__.py:24 in <module> │
│ │
│ 23 try: │
│ ❱ 24 from . import multiarray │
│ 25 except ImportError as exc: │
│ │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/multiarray.py:10 in <module> │
│ │
│ 9 import functools │
│ ❱ 10 from . import overrides │
│ 11 from . import _multiarray_umath │
│ │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/overrides.py:8 in <module> │
│ │
│ 7 from .._utils._inspect import getargspec │
│ ❱ 8 from numpy.core._multiarray_umath import ( │
│ 9 add_docstring, _get_implementing_args, _ArrayFunctionDispatcher) │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
╭─────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────────────────────╮
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__init__.py:130 in <module> │
│ │
│ 129 try: │
│ ❱ 130 from numpy.__config__ import show as show_config │
│ 131 except ImportError as e: │
│ │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__config__.py:4 in <module> │
│ │
│ 3 from enum import Enum │
│ ❱ 4 from numpy.core._multiarray_umath import ( │
│ 5 __cpu_features__, │
│ │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/__init__.py:50 in <module> │
│ │
│ 49 __version__, exc) │
│ ❱ 50 raise ImportError(msg) │
│ 51 finally: │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.11 from "/home/jw/store/src/text-generation-webui/installer_files/env/bin/python"
* The NumPy version is: "1.26.4"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
The above exception was the direct cause of the following exception:
╭─────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────────────────────╮
│ /home/jw/store/src/text-generation-webui/./server.py:6 in <module> │
│ │
│ 5 │
│ ❱ 6 import accelerate # This early import makes Intel GPUs happy │
│ 7 │
│ │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/accelerate/__init__.py:16 in <module> │
│ │
│ 15 │
│ ❱ 16 from .accelerator import Accelerator │
│ 17 from .big_modeling import ( │
│ │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/accelerate/accelerator.py:32 in <module> │
│ │
│ 31 │
│ ❱ 32 import torch │
│ 33 import torch.utils.hooks as hooks │
│ │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/__init__.py:1120 in <module> │
│ │
│ 1119 from ._tensor import Tensor │
│ ❱ 1120 from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_t │
│ 1121 │
│ │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/storage.py:14 in <module> │
│ │
│ 13 try: │
│ ❱ 14 import numpy as np │
│ 15 HAS_NUMPY = True │
│ │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__init__.py:135 in <module> │
│ │
│ 134 your python interpreter from there.""" │
│ ❱ 135 raise ImportError(msg) from e │
│ 136 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
(/home/jw/store/src/text-generation-webui/installer_files/env) ➜ text-generation-webui git:(main) ✗
System Info
GPU
NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0 |
NVIDIA GeForce RTX 4080Ti
OS
Debian GNU/Linux 12
Linux jw-box 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux
CPU
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
CPU(s): 16
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
MEMORY
49172584
Hi. I run text gen on 3 different Debian 12.5 servers. My first recommendation would be to upgrade your Nvidia driver. The 525 driver may not be your issue, but it will cause you to use more VRAM and may cause CUDA library issues since your driver only supports CUDA 12.0. See https://www.linuxcapable.com/install-nvidia-drivers-on-debian/ I've used 535, 54x and am now on 550 without issues.
My specific means for installing is:
cd src
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui/
./start_linux.sh --listen
I used to envelope the above in its own conda environment but that broke for installs awhile ago and I now run the above outside of any virtualized python env. My global python version is 3.11.5.
I'm also running commit 5b91dbb73bc96722314dc04479cd5a4e732fc9bc
It may be possible there was a very recent bug introduced in the last few commits. If updating your driver doesn't help, try doing a fresh install on that commit instead of what's most recent.
I don't know why auto installation is so attractive, I've never had a good experience with it on devices with multiple versions of python, use "Manual installation using Conda" from README.md.
PS: your version of python jumping from 3.10 to 3.11 and back again during a single process.
This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}