llama.cpp
llama.cpp copied to clipboard
How do I install with Make?
I wasn't able to run cmake on my system (ubuntu 20.04), but just wondering how I get the built binaries out, installed on the system...
make install
didn't work for me :(
I wasn't sure, not a C guy, but was trying my best to build ./examples/server using Make
You have successfully built and you want to execute this binary anywhere?
You can move the binary file to /usr/local/bin , and then you can execute the server anywhere
oh okay, but when I move to 'examples/server' I then run;
cmake .
make
and then I get the following;
➜ server git:(master) ✗ make
[ 50%] Linking CXX executable server
/usr/bin/ld: cannot find -lcommon
/usr/bin/ld: cannot find -lllama
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/server.dir/build.make:84: server] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/server.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
If I try;
export LD_LIBRARY_PATH="/home/user/Tools/06_MachineLearning/llama.cpp:$LD_LIBRARY_PATH"
I still get the same error :(
There are instructions in the README, but generally don't ever run cmake in the source directory.
CMake
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -D LLAMA_BUILD_SERVER=1
cd build
make -j4 server
It should put it into build/bin/server
inside the llama.cpp source dir.
GNU Make
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j4 LLAMA_BUILD_SERVER=1 server
Should create server
into the llama.cpp source directory.
There is no install target. Just copy the executable to where you use it.
I run llama.cpp server building with CMake, thanks @SlyEcho
Here's an example, first start server:
cd build
cd bin
./server -m ~/llama.cpp/models/airoboros-7b-gpt4-1.2.ggmlv3.q4_0.bin -c 2048 -t 3 -b 7 --alias DavidBowie
Then query with something like this,
curl --request POST --url http://127.0.0.1:8080/completion --data '{"prompt": "List 3 ways to have fun at the beach.","n_predict": 128}'
This issue was closed because it has been inactive for 14 days since being marked as stale.