How do I install with Make?

[chrisbward]

I wasn't able to run cmake on my system (ubuntu 20.04), but just wondering how I get the built binaries out, installed on the system...

make install

didn't work for me :(

[chrisbward]

I wasn't sure, not a C guy, but was trying my best to build ./examples/server using Make

[zhaijunxiao]

You have successfully built and you want to execute this binary anywhere?

You can move the binary file to /usr/local/bin , and then you can execute the server anywhere

[chrisbward]

oh okay, but when I move to 'examples/server' I then run;

cmake .
make

and then I get the following;

➜  server git:(master) ✗ make
[ 50%] Linking CXX executable server
/usr/bin/ld: cannot find -lcommon
/usr/bin/ld: cannot find -lllama
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/server.dir/build.make:84: server] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/server.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

[chrisbward]

If I try;

export LD_LIBRARY_PATH="/home/user/Tools/06_MachineLearning/llama.cpp:$LD_LIBRARY_PATH"

I still get the same error :(

[SlyEcho]

There are instructions in the README, but generally don't ever run cmake in the source directory.

CMake

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -D LLAMA_BUILD_SERVER=1
cd build
make -j4 server

It should put it into build/bin/server inside the llama.cpp source dir.

GNU Make

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j4 LLAMA_BUILD_SERVER=1 server

Should create server into the llama.cpp source directory.

There is no install target. Just copy the executable to where you use it.

[ghost]

I run llama.cpp server building with CMake, thanks @SlyEcho

Here's an example, first start server:

cd build
cd bin

./server -m ~/llama.cpp/models/airoboros-7b-gpt4-1.2.ggmlv3.q4_0.bin -c 2048 -t 3 -b 7 --alias DavidBowie

Then query with something like this, curl --request POST --url http://127.0.0.1:8080/completion --data '{"prompt": "List 3 ways to have fun at the beach.","n_predict": 128}'

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.