ERROR: Failed building wheel for llama-cpp-python

[Song367]

Describe the bug

install llama display error ERROR: Failed building wheel for llama-cpp-python

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

1. pip install -r requirements.txt
2. ERROR: Failed building wheel for llama-cpp-python

Screenshot

No response

Logs

Collecting llama-cpp-python==0.1.36
  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1b/ea/3f2aff10fd7195c6bc8c52375d9ff027a551151569c50e0d47581b14b7c1/llama_cpp_python-0.1.36.tar.gz (1.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in ./tenv/lib/python3.8/site-packages (from llama-cpp-python==0.1.36) (4.5.0)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [94 lines of output]

System Info

GPU

[innocentius]

Same error here, running on Ubuntu 18.04.

Wouldn't remove llama-cpp-python break something?

[Tom-Neverwinter]

is this an amd cpu?

not related to cpu.

[innocentius]

Error solved by upgrading to gcc-11. Try that first.

[xNul]

The same error here https://github.com/oobabooga/one-click-installers/issues/30#issuecomment-1517950800

[BlairSadewitz]

Error solved by upgrading to gcc-11. Try that first.

That's what I did, and the error resolved.

[LiemLin]

Error solved by upgrading to gcc-11. Try that first.

That's what I did, and the error resolved.

Is your operating system centos？

[maplessssy]

Error solved by upgrading to gcc-11. Try that first.

That's what I did, and the error resolved.

gcc-11 not work.

[madeepakkumar1]

upgrading gcc-11, did not work

[djdanielsson]

i am getting the same error and gcc-11 doesn't do anything. Ubuntu 22.04 and Fedora 37

[peitianyu]

Same error here, running on Ubuntu 18.04.

Wouldn't remove llama-cpp-python break something?

Have you solved the problem? im also running on Ubuntu 18.04.

[crearo]

Updating to gcc-11 and g++-11 worked for me on Ubuntu 18.04.

Did that using sudo apt install gcc-11 and sudo apt install g++-11.

[robicity]

Error solved by upgrading to gcc-11. Try that first.

Using it on windows WSL i had additionally make a few more installations:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update
sudo apt install gcc-11 g++-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 60 --slave /usr/bin/g++ g++ /usr/bin/g++-11
pip install --upgrade pip
pip install --upgrade setuptools wheel
sudo apt-get install build-essential

It was all done to install oobabooga on windows WSL. Here my complete list for a windows 10 NVIDIA System:

# Update Ubuntu packages
sudo apt update
sudo apt upgrade

# Download and install Miniconda
curl -sL "https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh" > "Miniconda3.sh"
bash Miniconda3.sh
rm Miniconda3.sh

# IMPORTANT - restart the terminal so it says (bash) in the beginning of the line

# Update conda, install wget, create and activate conda environment "textgen"
conda update conda
conda install wget
conda create -n textgen python=3.10.9
conda activate textgen

# Install CUDA libraries
pip3 install torch torchvision torchaudio

# Add PPA for gcc-11, update packages, install gcc-11, g++-11, update pip and setuptools, install build-essential
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update
sudo apt install gcc-11 g++-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 60 --slave /usr/bin/g++ g++ /usr/bin/g++-11
pip install --upgrade pip
pip install --upgrade setuptools wheel
sudo apt-get install build-essential

# Clone and setup oobabooga text-generation-webui
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt

# Final update of Ubuntu packages
sudo apt update
sudo apt upgrade

[tanhm12]

Updating to gcc-11 and g++-11 worked for me on Ubuntu 18.04.

Did that using sudo apt install gcc-11 and sudo apt install g++-11.

This should be the accepted solution. gcc-11 alone would not work, it needs both gcc-11 and g++-11. After installing the two above, run CXX=g++-11 CC=gcc-11 pip install -r requirements.txt and it should work (at least for me).

[itgoldman]

nothing worked until i ran this CMAKE_ARGS="-DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.48 (from pr #120)

[baphilia]

nothing worked until i ran this CMAKE_ARGS="-DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.48 (from https://github.com/imartinez/privateGPT/pull/120)

Thank you @itgoldman .
This worked on Windows (thanks chatgpt):

set "CMAKE_ARGS=-DLLAMA_OPENBLAS=on"
set "FORCE_CMAKE=1"
pip install llama-cpp-python --no-cache-dir

[parasharamit]

Worked for me on Ubuntu18.04

sudo apt install software-properties-common
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt install gcc-11 g++-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 90 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11
sudo apt-get update

pip install -r requirements.txt

[gonewilds]

this is work for me

[Tw0sheds]

@robicity with the save - build-essential was the package for me, but I also tried a few methods mentioned previously so they could help you:

sudo apt-get install build-essential sudo apt-get install gcc-11 g++-11

gcc11 or 12, it doesn't matter I don't think. with those installed you can rerun your pip command

[B0-B]

Hey everyone, I installed a fresh ubuntu and this sequence solved this issue:

Update apt package manager and change into home directory

sudo apt-get update && cd ~ 

Install pre-requisites

sudo apt install curl &&
sudo apt install cmake -y &&
sudo apt install python3-pip -y &&
pip3 install testresources # dependency for launchpadlib

Also gcc-11 and g++-11 need to be installed to overcome this llama-cpp-python compilation issue

sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test &&
sudo apt install -y gcc-11 g++-11 &&
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 60 --slave /usr/bin/g++ g++ /usr/bin/g++-11 &&
pip3 install --upgrade pip &&
pip3 install --upgrade setuptools wheel &&
sudo apt-get install build-essential &&
gcc-11 --version # check if gcc works

Download the WebUI installer from repository and unpack it

wget https://github.com/oobabooga/text-generation-webui/releases/download/installers/oobabooga_linux.zip &&
unzip oobabooga_linux.zip && 
rm oobabooga_linux.zip

change into the downloaded folder and run the installer, this will download the necessary files etc. into a single folder

cd oobabooga_linux &&
bash start_linux.sh

Hope this helps!

[DJJones66]

Hey everyone, I installed a fresh ubuntu and this sequence solved this issue:

Update apt package manager and change into home directory

sudo apt-get update && cd ~ 

Install pre-requisites

sudo apt install curl &&
sudo apt install cmake -y &&
sudo apt install python3-pip -y &&
pip3 install testresources # dependency for launchpadlib

Also gcc-11 and g++-11 need to be installed to overcome this llama-cpp-python compilation issue

sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test &&
sudo apt install -y gcc-11 g++-11 &&
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 60 --slave /usr/bin/g++ g++ /usr/bin/g++-11 &&
pip3 install --upgrade pip &&
pip3 install --upgrade setuptools wheel &&
sudo apt-get install build-essential &&
gcc-11 --version # check if gcc works

Download the WebUI installer from repository and unpack it

wget https://github.com/oobabooga/text-generation-webui/releases/download/installers/oobabooga_linux.zip &&
unzip oobabooga_linux.zip && 
rm oobabooga_linux.zip

change into the downloaded folder and run the installer, this will download the necessary files etc. into a single folder

cd oobabooga_linux &&
bash start_linux.sh

Hope this helps!

Perfect, flawless. Someone needs to add this to the docs

[DavidInRacine]

Same issue here in KDE Neon with GCC 11.3.0 and G++ 11.3.0, and also in Manjaro with GCC 12.2.x. In Manjaro oobabooga complained that it could find GCC 9 compiler. None of the solutions in this thread nor in the oobabooga Reddit thread titled 'Failed building wheel for llama-cpp-python' worked.

Curiously, I had no problem rolling oobabooga with all wheels attached in Linux Mint 21.1. I don't remember which compiler version is in Linux Mint 21.1, probably GCC 11.3.0. I did not have to jump through any hoops nor whisper sacred incantations while shaking a chicken foot and turning around three times with my eyes closed.

I don't think anyone really got to the bottom of this Llama-cpp-python wheel failure issue in a systematic way, especially when one Debian derivative works (Linux Mint) and another Debian variant (KDE Neon) does not.

15:49 - Edited to correct a typo and improve legibility.

[LeafmanZ]

godbless you this worked

nothing worked until i ran this CMAKE_ARGS="-DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.48 (from imartinez/privateGPT#120)

Thank you @itgoldman . This worked on Windows (thanks chatgpt):

set "CMAKE_ARGS=-DLLAMA_OPENBLAS=on"
set "FORCE_CMAKE=1"
pip install llama-cpp-python --no-cache-dir

[nightwalker89]

Updating to gcc-11 and g++-11 worked for me on Ubuntu 18.04. Did that using sudo apt install gcc-11 and sudo apt install g++-11.

This should be the accepted solution. gcc-11 alone would not work, it needs both gcc-11 and g++-11. After installing the two above, run CXX=g++-11 CC=gcc-11 pip install -r requirements.txt and it should work (at least for me).

It works for me, thank you

[atharvapatiil]

First, install conda install -c conda-forge cxx-compiler And then try running pip install llama-cpp-python==0.1.48 It worked for me. As it will pick c++ compiler from conda instead of root machine. So, without changing compiler version you will able to install lamma

[VapoZ]

For Windows:

1. make sure you're using Visual Studio 2022 and not just Visual Studio Code
2. Open the "x64 Native Tools Command Prompt for Visual Studio 2022" (u can search for it after pressing windows key)
3. run pip install llama-cpp-python==0.1.48
4. ...
5. profit

[ilkeasal]

atharvapatiil it worked! thanks

[DavidInRacine]

Same issue here in KDE Neon with GCC 11.3.0 and G++ 11.3.0, and also in Manjaro with GCC 12.2.x. In Manjaro oobabooga complained that it could find GCC 9 compiler. None of the solutions in this thread nor in the oobabooga Reddit thread titled 'Failed building wheel for llama-cpp-python' worked.

I am reporting back with success compiling and running oobabooga today in all three of my Linux distros – for the first time ever! These distros include: Linux Mint 21.1 (stable), KDE Neon (rolling plasma), and Manjaro KDE (rolling Arch). There are no more wheels problems and no more safetensors problems. All 13B models tested perfectly.

Well done!

[RandomAct5]

Updating to gcc-11 and g++-11 worked for me on Ubuntu 18.04.

Did that using sudo apt install gcc-11 and sudo apt install g++-11.

Thanks. This worked for me in Fedora 38. In Fedora, installing nvidia cuda driver toolkit has gcc in the pre-req.

sudo dnf install gcc-13 tab to autofill sudo dnf install gcc-c++.13 tab to autofill

then ran one-click again and it flew straight through to completion.

[PinkalShah2022]

Sorry guys, I am also getting the error of " Failed building wheel for llama-cpp-python Failed to build llama-cpp-python ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects"

I am on windows 11, is there any prerequisite installation to get rid of this error?

[igorMIA]

In my case the problem was related to the Ubuntu CUDA settings, I fixed it by setting up the CUDACXX param as follows: CUDACXX=/usr/local/cuda/bin/nvcc CMAKE_ARGS="-DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python