Failed to load dynamic library 'libllama.so'

[Vinayak006]

Description:

I encountered an issue while trying to run my Flutter application using the llama_cpp_dart package. The error occurs when the app attempts to load the libllama.so dynamic library. Below are the details:

Error Message:

Failed to load dynamic library '/libllama.so': dlopen failed: library "/libllama.so" not found
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
#2      Llama.lib (package:llama_cpp_dart/src/llama.dart:50:41)
#3      new Llama (package:llama_cpp_dart/src/llama.dart:79:5)
#4      LlamaProcessor._modelIsolateEntryPoint.<anonymous closure> (package:llama_cpp_dart/src/llama_processor.dart:122:21)
#5      _RootZone.runUnaryGuarded (dart:async/zone.dart:1594:10)
#6      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:365:11)
#7      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:297:7)
#8      _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:784:19)
#9      _StreamController._add (dart:async/stream_controller.dart:658:7)
#10     _StreamController.add (dart:async/stream_controller.dart:606:5)
#11     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

What I have done is:

1. Cloned llama.cpp and build libllama.so file.
2. Placed the libllama.so in root folder of the project and add the library path. Llama.libraryPath = "/llama_cpp_app/libllama.so";
3. And then I got add_subdirectory(./llama.cpp) issue, so I have did like below:

cd ~/.pub-cache/hosted/pub.dev/llama_cpp_dart-0.0.4
git clone https://github.com/ggerganov/llama.cpp.git

4. When I tried to load model:

llamaProcessor = LlamaProcessor(
                      path: _modelPathController.text,
                      modelParams: ModelParams(),
                      contextParams: ContextParams(),
                      samplingParams: SamplingParams(),
                    );

I got the above error: Failed to load dynamic library libllama.so

I dont know where exactly to place the file or Am I missing anything?

[netdur]

did you mean to use ./libllama.so instead of /libllama.so ?

[Vinayak006]

did you mean to use ./libllama.so instead of /libllama.so ?

I have my libllaam.so file in root folder. I have tried both:

// Using relative path from the lib directory
Llama.libraryPath = "../libllama.so";

Failed to load dynamic library '../libllama.so': dlopen failed: library "../libllama.so" not found

// using absolute path
Llama.libraryPath = "/home/user/projects/llama_cpp/libllama.so";

Failed to load dynamic library '/home/kingkong/YAVAR/projects/llama_cpp_app/libllama.so': dlopen failed: library "/home/kingkong/YAVAR/projects/llama_cpp_app/libllama.so" not found

I don't know what we are missing. Guide us on it @netdur

[ArachnaDigitalLtd]

Did you manage to solve this @Vinayak006 ?

I'm having this issue on both iOS and Android

[talelamira]

Had the same problem. I placed libllama.so in the root of the project and in lib folder I used Llama.libraryPath = '/libllama.so'; Don't know which one was called but I no longer have the error

[ndnam198]

I am having this issue on Android.

• My initial assumption was that because this package will build the library from source so I won't need have to include any "libllama.so" in my code manually. But it complains about "libllama.so" not found
• So I tried to put my build of llama.cpp for my specific Android device in different places:

1. In assets folder
2. In project root, next to pubspec.yaml
3. In lib folder and have set different paths for Llama.libraryPath but none works.

[ndnam198]

Update on the issue: Today I tried to clone llama_cpp_dart and link to my host project locally. And what I got was this new error:

LlamaException (LlamaException: Failed to initialize Llama (Invalid argument(s): Failed to load dynamic library 'libllama.so': dlopen failed: library "libggml.so" not found: needed by /data/app/~~cCNNT6K4mNaeSyOIrSkQHQ==/com.example.flutter_embedded_ai-ySMsC2j1ZqvZfbmp69sgfg==/base.apk!/lib/arm64-v8a/libllama.so in namespace clns-7))

I use APK Analyzer in AS, libllama.so shows up there just fine, but no libggml.so can be found. So I think the problem now should be lacking libggml.so from the final product

Please help to resolve this issue soon. Thanks

[rekire]

You might need to put the .so file into a LD_LIBRARY_PATH directory or extend it to contain your personal directory containing your libllama.so.

[TzeLun]

@rekire

Hi there! Appreciate your assistance in this repo. I had the problem in #51 so I tried your method in #54 to which I am currently facing the same error in this thread. I tried on both windows llama.dll and ubuntu libllama.so, via manually compiling or direct download from release, respectively. I tested those libraries through llama-cli and they worked. I also tried what others prescribed above.

I am currently running this from a dart script on my machine, I have yet to integrate this to my Android/IOS app (NOTE: I haven't tried this on Android/IOS just yet)

You might need to put the .so file into a LD_LIBRARY_PATH directory or extend it to contain your personal directory containing your libllama.so.

I would like to know more regarding your comment above as quoted. Can't think of any other possibilities as of now.

[rekire]

@TzeLun as you might have seen in my build.yaml I just use "override" the environment variable for the command. I run in the CI flutter test therefore I run LD_LIBRARY_PATH="$LD_LIBRARY_PATH:build/bin" flutter test since I unzip the files there.

LD_LIBRARY_PATH is an environment variable where the system looks for .so files. This "simply" works for me, you can print your directories with echo $LD_LIBRARY_PATH. If you want to do it "right" google for it I was too lazy to be honest :-)

[TzeLun]

Hi @rekire

Tried it and it works now for both ubuntu and windows (env variable path)! Thanks!

[DevMaan707]

Did you add the tag in AndroidManifest? android:extractNativeLibs="true" put this inside the <applications tag <application android:label="example" android:name="${applicationName}" android:extractNativeLibs="true" ..