llama-cli: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

Name and Version

Version: llama-b4434-bin-ubuntu-x64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-cli

Command line

[erick@Xeon bin]$  ./llama-cli -m ~/Apps/Llama/EVA-LLaMA-3.33-70B-v0.1.Q4_K_M.gguf -p "I believe the meaning of life is" -n 128

Problem description & steps to reproduce

Trying to run this new version and got this message: [erick@Xeon bin]$ ./llama-cli -m ~/Apps/Llama/EVA-LLaMA-3.33-70B-v0.1.Q4_K_M.gguf -p "I believe the meaning of life is" -n 128 ./llama-cli: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

The last build version that works properly is: llama-b4406-bin-ubuntu-x64

First Bad Commit

The last build version that works properly is: llama-b4406-bin-ubuntu-x64

Relevant log output

No response

Did you locate the path to libllama.so in your system after the installation and tried to append this path to LD_LIBRARY_PATH?

Got the same by using the binary files. What do you mean with installation? Should i not work out of the box with binary files?

So when you build it locally it will work I'll just go this way

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp


# https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
# CUDA BUILD
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

cd build/bin
./llama-cli -m "/home/user/test/xxxxxxxxxx.gguf"

Last builds do not include the .so library anymore. Related: https://github.com/ggerganov/llama.cpp/issues/11144

Can confirm Ubuntu zip files do not include libllama.so anymore. As a workaround you can either compile it yourself or get the libraries from the Docker version.

Will be fixed with the next release: #11392

Please close the issue when you confirm.

Will be fixed with the next release: #11392

Please close the issue when you confirm.

Have tested b4552, it works on my machine(s). Output of ldd llama-cli on Ubuntu 24.04.1:

 ldd llama-cli
        linux-vdso.so.1 (0x00007fff8b081000)
        libllama.so (0x00007fdfde534000)
        libggml.so (0x00007fdfde525000)
        libggml-base.so (0x00007fdfde451000)
        libcurl.so.4 => /lib/x86_64-linux-gnu/libcurl.so.4 (0x00007fdfde383000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fdfde105000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdfde01a000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fdfddfec000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdfdddda000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fdfde801000)
        libggml-cpu.so (0x00007fdfddd22000)
        libggml-rpc.so (0x00007fdfddd0e000)
        libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007fdfddce1000)
        libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007fdfddcbf000)
        librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007fdfddca1000)
        libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007fdfddc30000)
        libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007fdfddc1c000)
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007fdfddb72000)
        libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fdfdd65d000)
        libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fdfdd609000)
        libldap.so.2 => /lib/x86_64-linux-gnu/libldap.so.2 (0x00007fdfdd5ac000)
        liblber.so.2 => /lib/x86_64-linux-gnu/liblber.so.2 (0x00007fdfdd59c000)
        libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fdfdd4e2000)
        libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007fdfdd4d4000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fdfdd4b6000)
        libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fdfdd460000)
        libunistring.so.5 => /lib/x86_64-linux-gnu/libunistring.so.5 (0x00007fdfdd2b3000)
        libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007fdfdd0b9000)
        libhogweed.so.6 => /lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007fdfdd071000)
        libnettle.so.8 => /lib/x86_64-linux-gnu/libnettle.so.8 (0x00007fdfdd01a000)
        libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fdfdcf96000)
        libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fdfdcecd000)
        libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fdfdcea1000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fdfdce9b000)
        libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fdfdce8c000)
        libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fdfdce72000)
        libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007fdfdce4f000)
        libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fdfdccab000)
        libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007fdfdcc95000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fdfdcc8c000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fdfdcc79000)
        libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007fdfdcc6d000)

b4641

./build/bin/llama-cli ./build/bin/llama-cli: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

system: Ubuntu 18

b4641

./build/bin/llama-cli ./build/bin/llama-cli: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

system: Ubuntu 18

@Liozhang you have to go into the bin directory to help it find the library:

cd build/bin
./llama-cli

or configure your dynamic loader to find the library in that directory

For people like me who want to package the binary version for their distro, the fix was simply to install the libraries into the system-wide shared library path.

Example adapted from the AUR package I've put together:

sudo install -Dm755 build/bin/lib* -t /usr/lib/

For a concise and helpful intro to shared libraries, see this chapter from the LPI 101 learning material.

For people like me who want to package the binary version for their distro, the fix was simply to install the libraries into the system-wide shared library path.

Example adapted from the AUR package I've put together:

sudo install -Dm755 build/bin/lib* -t /usr/lib/

For a concise and helpful intro to shared libraries, see this chapter from the LPI 101 learning material.

If you are not creating a package, it is better to not mess with the directories where the package manager is putting their stuff, and use /usr/local/lib

This issue was closed because it has been inactive for 14 days since being marked as stale.