Error loading gemma gpu model in latest sample code android

[affereaflaw]

Testing device:

Pixel 4

Testing model:

gemma-2b-it-gpu-int4.bin and other gpu models

Log:

Load model error: internal: Failed to initialize engine: %sUNKNOWN: Failed to build program executable - Build program failureBC-src-code:53:18: error: no matching builtin function for call to 'sub_group_broadcast' ... ^ 2025-03-06 15:44:54.372 17164-17239 com.google...renceModel com...diapipe.examples.llminference E BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'unsigned long' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ BC-src-code:74:18: error: no matching builtin function for call to 'sub_group_broadcast' r1 += src1.y * sub_group_broadcast(weights_cache_wave_var2, 5u); ^~~~~~~~~~~~~~~~~~~ BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'int' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'unsigned int' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'long' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'unsigned long' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ BC-src-code:75:18: error: no matching builtin function for call to BC-src-code:53:18: note: candidate function not viable: no known conversion from 'half4' to 'unsigned long' for 1st argument; r0 += src0.x * sub_group_broadcast(weights_cache_wave_var0, 0u); ^ 32 diagnostic(s) generated.

                                                                                                	at com.google.mediapipe.tasks.core.LlmTaskRunner.nativeCreateEngine(Native Method)
                                                                                                	at com.google.mediapipe.tasks.core.LlmTaskRunner.<init>(LlmTaskRunner.java:168)

2025-03-06 15:44:54.373 17164-17239 com.google...renceModel com...diapipe.examples.llminference E at com.google.mediapipe.tasks.genai.llminference.LlmInference.(LlmInference.java:118) at com.google.mediapipe.tasks.genai.llminference.LlmInference.createFromOptions(LlmInference.java:84) at com.google.mediapipe.examples.llminference.InferenceModel.(InferenceModel.kt:51) at com.google.mediapipe.examples.llminference.InferenceModel.(Unknown Source:0) at com.google.mediapipe.examples.llminference.InferenceModel$Companion.resetInstance(InferenceModel.kt:84) at com.google.mediapipe.examples.llminference.LoadingScreenKt$LoadingRoute$4$1.invokeSuspend(LoadingScreen.kt:75) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:100) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

[pazhaniapa]

I am facing the same issue, if we copy the model downloaded from kaggle https://www.kaggle.com/models/google/gemma-2/tfLite/gemma2-2b-it-gpu-int8, not able to load the model, it says the below error: Load model error: internal: Failed to initialize engine: %sINVALID_ARGUMENT: It's no longer supported to load from directory, please consider convert it to a single-file asset. com.google.mediapipe.framework.MediaPipeException: internal: Failed to initialize engine: %sINVALID_ARGUMENT: It's no longer supported to load from directory, please consider convert it to a single-file asset. at com.google.mediapipe.tasks.core.LlmTaskRunner.nativeCreateEngine(Native Method) at com.google.mediapipe.tasks.core.LlmTaskRunner.(LlmTaskRunner.java:168) at com.google.mediapipe.tasks.genai.llminference.LlmInference.(LlmInference.java:118) at com.google.mediapipe.tasks.genai.llminference.LlmInference.createFromOptions(LlmInference.java:84) at com.google.mediapipe.examples.llminference.InferenceModel.(InferenceModel.kt:52) at com.google.mediapipe.examples.llminference.InferenceModel.(Unknown Source:0) at com.google.mediapipe.examples.llminference.InferenceModel$Companion.resetInstance(InferenceModel.kt:85) at com.google.mediapipe.examples.llminference.LoadingScreenKt$LoadingRoute$4$1.invokeSuspend(LoadingScreen.kt:75) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)

[huangzhengxiang]

@affereaflaw Me too encountered the same issue. It seems to be an issue related to LiteRT OpenCL backend compilation.

[huangzhengxiang]

My test device is with Snapdragon 8 Gen 3 core.

[affereaflaw]

@huangzhengxiang it was all normal for genai 0.10.20 but when i updated to v21, gpu models wouldn't run

[huangzhengxiang]

@affereaflaw Yes! same for me!