llama_cpp_dart icon indicating copy to clipboard operation
llama_cpp_dart copied to clipboard
verified publisher icon netdur

error: Cannot invoke native callback from a different isolate.

Open hg0428 opened this issue 1 month ago • 3 comments 

flutter: [LlamaService] Model disposed
flutter: [LlamaService] Initializing model: /Users/hudsongouge/Projects/aichatapp/asset/model/gguf/Qwen3-4B-Q4_K_M.gguf
flutter: [LlamaService] Using library from app bundle: /Users/hudsongouge/Projects/aichatapp/build/macos/Build/Products/Debug/aichatapp.app/Contents/Frameworks/libllama.dylib
flutter: [MyAppState] Initializing Qwen model: Qwen3-4B
flutter: [LlamaService] Using model from dev path: /Users/hudsongouge/Projects/aichatapp/asset/model/gguf/Qwen3-4B-Q4_K_M.gguf
flutter: [LlamaService] Model disposed
flutter: [LlamaService] Initializing model: /Users/hudsongouge/Projects/aichatapp/asset/model/gguf/Qwen3-4B-Q4_K_M.gguf
flutter: [LlamaService] Using library from app bundle: /Users/hudsongouge/Projects/aichatapp/build/macos/Build/Products/Debug/aichatapp.app/Contents/Frameworks/libllama.dylib
../../../flutter/third_party/dart/runtime/vm/runtime_entry.cc: 5037: error: Cannot invoke native callback from a different isolate.
version=3.10.1 (stable) (Tue Nov 18 11:02:31 2025 -0800) on "macos_arm64"
pid=38416, thread=129183, isolate_group=main(0xa3a45ce00), isolate=IsolateChild.registerWithParent(0xa40107100)
os=macos, arch=arm64, comp=no, sim=no
isolate_instructions=10892ace0, vm_instructions=10892ace0
fp=16ed713a0, sp=16ed71360, pc=1086a6298
  pc 0x00000001086a6298 fp 0x000000016ed713a0 dart::Profiler::DumpStackTrace(bool)+0x30
  pc 0x0000000108546d10 fp 0x000000016ed713c0 dart::Assert::Fail(char const*, ...) const+0x24
  pc 0x00000001086c1f5c fp 0x000000016ed71450 DLRT_GetFfiCallbackMetadata+0x3b0
  pc 0x0000000115d23f10 fp 0x000000016ed71468 Unknown symbol
  pc 0x000000013bcc65d0 fp 0x000000016ed71550 llama_log_internal(ggml_log_level, char const*, ...)+0x74
  pc 0x000000013bcc65d0 fp 0x000000016ed71640 llama_log_internal(ggml_log_level, char const*, ...)+0x74
  pc 0x000000013bcc8164 fp 0x000000016ed71820 llama_kv_cache::llama_kv_cache(llama_model const&, ggml_type, ggml_type, bool, bool, bool, unsigned int, unsigned int, unsigned int, unsigned int, llama_swa_type, std::__1::function<bool (int)> const&, std::__1::function<int (int)> const&)+0xc70
  pc 0x000000013bd16724 fp 0x000000016ed718e0 llama_model::create_memory(llama_memory_params const&, llama_cparams const&) const+0x230
  pc 0x000000013bc9d8c4 fp 0x000000016ed71a10 llama_context::llama_context(llama_model const&, llama_context_params)+0xb58
  pc 0x000000013bca5404 fp 0x000000016ed71ae0 llama_init_from_model+0x21c
  pc 0x000000013bca54f4 fp 0x000000016ed71b70 llama_new_context_with_model+0x3c
  pc 0x0000000122a87fc8 fp 0x000000016ed71c10 Unknown symbol
  pc 0x000000013a89fb04 fp 0x000000016ed71c58 Unknown symbol
  pc 0x000000013a89f458 fp 0x000000016ed71cb8 Unknown symbol
  pc 0x000000013a8b5c90 fp 0x000000016ed71d78 Unknown symbol
  pc 0x000000013a8b4820 fp 0x000000016ed71e48 Unknown symbol
  pc 0x000000013a8d3f2c fp 0x000000016ed71ee8 Unknown symbol
  pc 0x000000013a8cfce4 fp 0x000000016ed71fa8 Unknown symbol
  pc 0x000000013a8cf570 fp 0x000000016ed71fe8 Unknown symbol
  pc 0x00000001348c46cc fp 0x000000016ed72068 Unknown symbol
  pc 0x000000013a8cced4 fp 0x000000016ed720f8 Unknown symbol
  pc 0x000000013a8cca60 fp 0x000000016ed72138 Unknown symbol
  pc 0x00000001348c46cc fp 0x000000016ed721b8 Unknown symbol
  pc 0x0000000139da6110 fp 0x000000016ed72210 Unknown symbol
  pc 0x0000000139da58a8 fp 0x000000016ed72250 Unknown symbol
  pc 0x000000013a8cc190 fp 0x000000016ed72298 Unknown symbol
  pc 0x000000013a8cb938 fp 0x000000016ed722d8 Unknown symbol
  pc 0x000000013a8cb488 fp 0x000000016ed72320 Unknown symbol
  pc 0x000000013a8f4d70 fp 0x000000016ed72360 Unknown symbol
  pc 0x00000001348c46cc fp 0x000000016ed723e0 Unknown symbol
  pc 0x0000000139da6110 fp 0x000000016ed72438 Unknown symbol
  pc 0x0000000139da58a8 fp 0x000000016ed72478 Unknown symbol
  pc 0x0000000139da5508 fp 0x000000016ed724b8 Unknown symbol
  pc 0x000000013a13c640 fp 0x000000016ed724f8 Unknown symbol
  pc 0x0000000139da5138 fp 0x000000016ed72538 Unknown symbol
  pc 0x0000000139dcf5c0 fp 0x000000016ed72578 Unknown symbol
  pc 0x000000012b0b948c fp 0x000000016ed725e0 Unknown symbol
  pc 0x000000012b0b8874 fp 0x000000016ed72638 Unknown symbol
  pc 0x0000000122a83a64 fp 0x000000016ed72710 Unknown symbol
  pc 0x000000010859a184 fp 0x000000016ed72790 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&)+0x1d4
  pc 0x000000010859c294 fp 0x000000016ed727e0 dart::DartLibraryCalls::HandleMessage(long long, dart::Instance const&)+0x144
  pc 0x00000001085c37c0 fp 0x000000016ed72d90 dart::IsolateMessageHandler::HandleMessage(std::_fl::unique_ptr<dart::Message, std::_fl::default_delete<dart::Message>>)+0x308
  pc 0x00000001085eedf0 fp 0x000000016ed72e10 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x144
  pc 0x00000001085ef34c fp 0x000000016ed72e70 dart::MessageHandler::TaskCallback()+0x200
  pc 0x00000001086fa924 fp 0x000000016ed72f10 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x11c
  pc 0x00000001086faba8 fp 0x000000016ed72f50 dart::ThreadPool::Worker::Main(unsigned long)+0x88
  pc 0x00000001086a2c60 fp 0x000000016ed72fc0 dart::ThreadStart(void*)+0xd0
  pc 0x0000000192471c08 fp 0x000000016ed72fe0 _pthread_start+0x88
-- End of DumpStackTrace
  pc 0x0000000000000000 fp 0x000000016ed71c10 sp 0x0000000000000000 Cannot find code object+0x0
  pc 0x000000013a89fb04 fp 0x000000016ed71c58 sp 0x000000016ed71c20 [Optimized] llama_cpp.init:_llama_new_context_with_model@920190180.#ffiClosure665+0x134
  pc 0x000000013a89f458 fp 0x000000016ed71cb8 sp 0x000000016ed71c68 [Unoptimized] llama_cpp.llama_new_context_with_model+0x168
  pc 0x000000013a8b5c90 fp 0x000000016ed71d78 sp 0x000000016ed71cc8 [Unoptimized] Llama._initializeLlama@905097740+0x9c0
  pc 0x000000013a8b4820 fp 0x000000016ed71e48 sp 0x000000016ed71d88 [Unoptimized] new Llama.+0xd90
  pc 0x000000013a8d3f2c fp 0x000000016ed71ee8 sp 0x000000016ed71e58 [Unoptimized] LlamaChild._handleLoad@910094656+0x14c
  pc 0x000000013a8cfce4 fp 0x000000016ed71fa8 sp 0x000000016ed71ef8 [Unoptimized] LlamaChild.onData+0x724
  pc 0x000000013a8cf570 fp 0x000000016ed71fe8 sp 0x000000016ed71fb8 [Unoptimized] LlamaChild.onData+0xb0
  pc 0x00000001348c46cc fp 0x000000016ed72068 sp 0x000000016ed71ff8 [Unoptimized] [email protected]+0x1ec
  pc 0x000000013a8cced4 fp 0x000000016ed720f8 sp 0x000000016ed72078 [Unoptimized] CastStreamSubscription._onData@10040228+0x424
  pc 0x000000013a8cca60 fp 0x000000016ed72138 sp 0x000000016ed72108 [Unoptimized] CastStreamSubscription._onData@10040228+0xb0
  pc 0x00000001348c46cc fp 0x000000016ed721b8 sp 0x000000016ed72148 [Unoptimized] [email protected]+0x1ec
  pc 0x0000000139da6110 fp 0x000000016ed72210 sp 0x000000016ed721c8 [Unoptimized] _BufferingStreamSubscription@5048458._sendData@5048458+0x440
  pc 0x0000000139da58a8 fp 0x000000016ed72250 sp 0x000000016ed72220 [Unoptimized] _BufferingStreamSubscription@5048458._add@5048458+0x338
  pc 0x000000013a8cc190 fp 0x000000016ed72298 sp 0x000000016ed72260 [Unoptimized] _SyncBroadcastStreamController@5048458._sendData@5048458+0x500
  pc 0x000000013a8cb938 fp 0x000000016ed722d8 sp 0x000000016ed722a8 [Unoptimized] [email protected]+0x298
  pc 0x000000013a8cb488 fp 0x000000016ed72320 sp 0x000000016ed722e8 [Unoptimized] [email protected]+0x368
  pc 0x000000013a8f4d70 fp 0x000000016ed72360 sp 0x000000016ed72330 [Unoptimized] [email protected]+0xb0
  pc 0x00000001348c46cc fp 0x000000016ed723e0 sp 0x000000016ed72370 [Unoptimized] [email protected]+0x1ec
  pc 0x0000000139da6110 fp 0x000000016ed72438 sp 0x000000016ed723f0 [Unoptimized] _BufferingStreamSubscription@5048458._sendData@5048458+0x440
  pc 0x0000000139da58a8 fp 0x000000016ed72478 sp 0x000000016ed72448 [Unoptimized] _BufferingStreamSubscription@5048458._add@5048458+0x338
  pc 0x0000000139da5508 fp 0x000000016ed724b8 sp 0x000000016ed72488 [Unoptimized] _SyncStreamController@5048458._sendData@5048458+0x218
  pc 0x000000013a13c640 fp 0x000000016ed724f8 sp 0x000000016ed724c8 [Unoptimized] _StreamController@5048458._add@5048458+0x240
  pc 0x0000000139da5138 fp 0x000000016ed72538 sp 0x000000016ed72508 [Unoptimized] [email protected]+0x298
  pc 0x0000000139dcf5c0 fp 0x000000016ed72578 sp 0x000000016ed72548 [Unoptimized] [email protected]+0xb0
  pc 0x000000012b0b948c fp 0x000000016ed725e0 sp 0x000000016ed72588 [Unoptimized] [email protected]:call+0xadc
  pc 0x000000012b0b8874 fp 0x000000016ed72638 sp 0x000000016ed725f0 [Unoptimized] _RawReceivePort@1026248._handleMessage@1026248+0x214
  pc 0x0000000122a83a64 fp 0x000000016ed72710 sp 0x000000016ed72648 [Stub] InvokeDartCode+0xd4
Lost connection to device.

I had no idea what this error was so I loaded it up in Cursor and asked Claude 4.5 Opus Thinking and it said it's a problem with the package. It said the issue was that you were handling logging wrong.

hg0428 avatar Nov 30 '25 15:11 hg0428

It said in Llama.dart on line 157 to 161 change it from

if (_verbos == false) {
      final nullCallbackPointer =
          Pointer.fromFunction<LlamaLogCallback>(Llama.llamaLogCallbackNull);
      lib.llama_log_set(nullCallbackPointer, nullptr);
    }

to

if (_verbos == false) {
      // Passing a Dart function pointer here causes crashes when llama.cpp
      // invokes the callback from threads owned by other isolates. Suppress
      // logging by providing a real null pointer so no Dart code runs.
      lib.llama_log_set(nullptr.cast(), nullptr);
    }

And that fix seemed to make it not crash, so I believe it.

hg0428 avatar Nov 30 '25 15:11 hg0428

thank you for pointing to bug and providing solution, this is indeed a recent changes as I am trying to figure out gpu on android

netdur avatar Nov 30 '25 15:11 netdur

Hope that helps. I had no idea lol. I just started using this package. Good thing Claude figured it out. Would have probably taken me an hour or more myself.

hg0428 avatar Nov 30 '25 15:11 hg0428