mediapipe icon indicating copy to clipboard operation
mediapipe copied to clipboard

Published 20 hours ago • verified publisher icon googlesamples

Is Gemma on device really this slow ?

Open MJ1998 opened this issue 2 months ago • 4 comments

I used llm_inference sample with gemma-2b-it-cpu-int4.bin on Pixel 8 Pro emulator.

The prefill speed seems to be in minutes.

Pixel 8 Pro configurations:- RAM - 22GB, VM heap - 512mb

Reference video https://github.com/googlesamples/mediapipe/assets/22965002/c7730dba-48e8-4eec-ae68-fe847d2778f2

MJ1998 avatar Apr 30 '24 14:04 MJ1998