Compile bug: iOS version able to build not not able to run

[Animaxx]

Git commit

https://github.com/ggerganov/llama.cpp/commit/43ed389a3f102517e6f7d5620d8e451e88afbf27

Operating systems

Mac

GGML backends

Metal

Problem description & steps to reproduce

related to https://github.com/ggerganov/llama.cpp/issues/10747

I have follow with the CI action for macOS

mkdir build
cd build

cmake -G Xcode .. \
    -DGGML_METAL_USE_BF16=ON \
    -DGGML_METAL_EMBED_LIBRARY=ON \
    -DLLAMA_BUILD_EXAMPLES=OFF \
    -DLLAMA_BUILD_TESTS=OFF \
    -DLLAMA_BUILD_SERVER=OFF \
    -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" \
    -DCMAKE_SYSTEM_NAME=Darwin \
    -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 \
    -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=ggml
    
cmake --build . --config Release -j $(sysctl -n hw.logicalcpu) -- CODE_SIGNING_ALLOWED=NO
sudo cmake --install . --config Release

# build for swift package
cd ..
xcodebuild -scheme llama-Package -destination 'generic/platform=macOS'

or build for iOS

mkdir build
cd build
cmake -G Xcode .. \
    -DGGML_METAL_USE_BF16=ON \
    -DGGML_METAL_EMBED_LIBRARY=ON \
    -DLLAMA_BUILD_EXAMPLES=OFF \
    -DLLAMA_BUILD_TESTS=OFF \
    -DLLAMA_BUILD_SERVER=OFF \
    -DCMAKE_SYSTEM_NAME=iOS \
    -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0 \
    -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=ggml

cmake --build . --config Release -j $(sysctl -n hw.logicalcpu) -- CODE_SIGNING_ALLOWED=NO
sudo cmake --install . --config Release

# build for swift package
cd ..
xcodebuild -scheme llama-Package -destination 'generic/platform=iOS'

the build is success but when run it shows error dyld[70862]: Library not loaded: @rpath/libggml.dylib

First Bad Commit

No response

Relevant log output

dyld[70862]: Library not loaded: @rpath/libggml.dylib
  Referenced from: <4A6AE769-FE07-3AAE-84F0-552992A8720C> /Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/LocalLLM.debug.dylib
  Reason: tried: '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/libggml.dylib' (no such file), '/usr/lib/system/introspection/libggml.dylib' (no such file, not in dyld cache), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/Frameworks/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/Frameworks/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/PackageFrameworks/libggml.dylib' (no such file), '/Users/animax/Library/Developer/Xcode/DerivedData/LocalLLM-ctyejtpossqslxdwvdukottbyflj/Build/Products/Debug/LocalLLM.app/Contents/MacOS/Frameworks/libggml.dylib' (no such file)
Message from debugger: Terminated due to signal 6

[Animaxx]

update: looks like dylib files are not able to copy into the product folder while the app build. I used this script to build for macOS version

mkdir build
cd build
cmake -G Xcode .. \
  -DCMAKE_INSTALL_RPATH="@rpath" \
  -DCMAKE_INSTALL_PREFIX="PRODUCT/macOS" \
  -DGGML_METAL_USE_BF16=ON \
  -DGGML_METAL_EMBED_LIBRARY=ON \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64"
cmake --build . --config Release -j $(sysctl -n hw.logicalcpu)
cmake --install . --config Release

cd ..
xcodebuild -scheme llama-Package -destination 'generic/platform=macOS'

and based on @ggerganov suggestion https://github.com/ggerganov/llama.cpp/issues/10747#issuecomment-2538203367 to add the search path the problem still not solve.

also tried add all "PRODUCT/macOS/lib" files in Build Phases -> new copy files phases, still not luck. but manually copy paste these files works.

I am trying to see if there any better way to do it, please let me know if any advice

[edisonzf2020]

It is recommended to restore the maintenance of Package.swift, which is very convenient for encapsulating other Swift projects.

[Animaxx]

mkdir build
cd build

echo "=== cmake for iOS"
cmake -G Xcode .. \
  -DCMAKE_INSTALL_RPATH="@rpath" \
  -DGGML_METAL_USE_BF16=ON \
  -DGGML_METAL_EMBED_LIBRARY=ON \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DCMAKE_SYSTEM_NAME=iOS \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=16.0 \
  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=ggml

cmake --build . --config Release -j $(sysctl -n hw.logicalcpu) -- CODE_SIGNING_ALLOWED=NO

cmake --install . --prefix ../../llama_product_ios

cd ..
xcodebuild -scheme llama-Package -destination 'generic/platform=iOS'

with set the search paths and have "copy file phases" for all the "lib" dylib as Frameworks section works for both iOS and MacOS for real device, but it not build for simulator.

[Tongziqi]

Have you solved this problem?

[Animaxx]

Have you solved this problem?

no.. it might work for debug but not for app store release. I am thinking maybe swift package is not compatible for this case

[pgorzelany]

@Animaxx sorry to bother you but where you able to find a solution?

[Animaxx]

No, I don't have a good solution for this one yet. the dylib not friendly to iOS. Maybe we can find a way to integrate the C++ code different iOS project? We need someone who extreme good at C++ CMake and iOS framework here...

[benliong]

mkdir build
cd build

echo "=== cmake for iOS"
cmake -G Xcode .. \
  -DCMAKE_INSTALL_RPATH="@rpath" \
  -DGGML_METAL_USE_BF16=ON \
  -DGGML_METAL_EMBED_LIBRARY=ON \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DCMAKE_SYSTEM_NAME=iOS \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=16.0 \
  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=ggml

cmake --build . --config Release -j $(sysctl -n hw.logicalcpu) -- CODE_SIGNING_ALLOWED=NO

cmake --install . --prefix ../../llama_product_ios

cd ..
xcodebuild -scheme llama-Package -destination 'generic/platform=iOS'

with set the search paths and have "copy file phases" for all the "lib" dylib as Frameworks section works for both iOS and MacOS for real device, but it not build for simulator.

for anyone interested, after cmake -G Xcode .. , open llama.cpp.xcodeproj and make sure that the development team is set for all targets, and then you won't need the -- CODE_SIGNING_ALLOWED=NO for cmake --build . --config Release -j $(sysctl -n hw.logicalcpu) -- CODE_SIGNING_ALLOWED=NO isn't necessary, and ultimately you'll need to have those signed if you want the app to be on the app store.

Thank you @Animaxx for documenting this. It works well for me.

[danemadsen]

For me llama.cpp is compiled by a podspec. The build system is copying the library files to the correct directory and flutter is loading the library correctly but none of the functions seem to do anything and return nullptr.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.