mediapipe
mediapipe copied to clipboard
Published
20 hours ago •
googlesamples
googlesamples
Not generate text with gemma-2b model
Unable to successfully perform inference tasks on Google Pixel 4 device, the error message is as follows:
17:04:57.271 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:04:57.452 OnBack...llback W OnBackInvokedCallback is not enabled for the application.
Set 'android:enableOnBackInvokedCallback="true"' in the application manifest.
17:04:57.483 Insets...roller D show(ime(), fromIme=true)
17:05:01.814 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:05:01.823 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:05:01.857 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:05:01.900 Remote...onImpl W getSurroundingText on inactive InputConnection
17:05:02.021 libc A Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x777a7e4000 in tid 6633 (mediapipe/6633), pid 6596 (es.llminference)
17:05:02.047 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:05:02.062 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
17:05:02.068 Insets...roller D show(ime(), fromIme=true)
17:05:02.118 Remote...onImpl W requestCursorAnchorInfo on inactive InputConnection
---------------------------- PROCESS STARTED (6666) for package com.google.mediapipe.examples.llminference ----------------------------
17:05:02.369 DEBUG A Cmdline: com.google.mediapipe.examples.llminference
17:05:02.369 DEBUG A pid: 6596, tid: 6633, name: mediapipe/6633 >>> com.google.mediapipe.examples.llminference <<<
17:05:02.369 DEBUG A #00 pc 00000000003e1614 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #01 pc 000000000037f650 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #02 pc 00000000003ff028 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #03 pc 00000000004025b8 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #04 pc 00000000003fedac /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #05 pc 00000000003800d4 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #06 pc 000000000036a868 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #07 pc 00000000000e088c /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #08 pc 00000000000cdb40 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #09 pc 00000000000bd038 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #10 pc 0000000000458408 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #11 pc 000000000043fd7c /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #12 pc 000000000043fa48 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #13 pc 00000000004544d0 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
17:05:02.369 DEBUG A #14 pc 0000000000454074 /data/app/~~KDGLoEYpcCDWDrQ8azrhyg==/com.google.mediapipe.examples.llminference-0NHsZ8brHDRoxg0ltksmYg==/base.apk!libllm_inference_engine_jni.so
---------------------------- PROCESS ENDED (6596) for package com.google.mediapipe.examples.llminference ----------------------------
---------------------------- PROCESS ENDED (6666) for package com.google.mediapipe.examples.llminference ----------------------------
Me, too. Device: Pixel 7 Pro Android14 Model: gemma-2b-it-cpu-int4.bin / gemma-1.1-2b-it-cpu-int4.bin
However, the gemma-2b-it-gpu-int4.bin works
I hope you took latest changes of llminference from the examples.
https://github.com/google-ai-edge/mediapipe-samples/blob/main/examples/llm_inference/android/app/src/main/java/com/google/mediapipe/examples/llminference/InferenceModel.kt
Can you measure the memory usage during the inference call? We are working on expanding our hardware support, but we are currently mostly focused on bringing inference to recent set of Android phones that provide more memory. If you are targeting Pixel 4, I would suggest using a smaller LLM such as Falcon 1B or the even smaller models outlined here: https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/examples/smollm
