llama.cpp
llama.cpp copied to clipboard
Missing ggml-metal.metal file during shared library build
When building llama.cpp
with Metal support on my Mac M1, the ggml-metal.metal
file is placed in the bin
directory correctly. However, when building it as a shared library: the pathForResource
method from ggml-metal.m
only looks in the directory where the .dylib file is located and fails to find the ggml-metal.metal
file, since it's searching in the wrong place.
Environment
- Hardware: Apple M1 Arm64, 8GB RAM
- Operating System: macOS Sonoma 14.3
- Kernel Version: Darwin 23.3.0
- CMake Version: 3.28.2
- Clang Version: 17.0.6
Steps to Reproduce
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON ..
cmake --build . --config Release
curl -LO https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf
./bin/main -m llama-2-7b-chat.Q5_K_M.gguf -p "Hi you how are you" -n 50 -e -ngl 33 -t 4
Log
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M1
ggml_metal_init: picking default device: Apple M1
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x600000c421c0 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
llama_new_context_with_model: failed to initialize Metal backend
llama_init_from_gpt_params: error: failed to create context with model 'openhermes-2.5-mistral-7b.Q5_K_M.gguf'
main: error: unable to load model
Is there any workaround in the meantime?
Have you tried building with cmake -DLLAMA_METAL_EMBED_LIBRARY=ON
I suddenly had the same problem after doing a pull.
make LLAMA_METAL_EMBED_LIBRARY=ON libllama.so solved the problem for me.
This issue was closed because it has been inactive for 14 days since being marked as stale.
paasing -ngl 0 resolved issue for me as I don't have GPU on my system