llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Eval bug: image encode time slow on mobile device

Open perp opened this issue 1 week ago • 0 comments

Name and Version

Model:NexaAIDev/OmniVLM-968M In function clip_image_batch_encode, it takes about 18 seconds to encode the image? far more beyond the llm part ,it task about 1.5 seconds. ggml_compute_forward compute the model graph about 1102 nodes, how to accelerate? Deivice: Android, SamSung S23

Image

Operating systems

Other? (Please let us know in description)

GGML backends

CPU

Hardware

Android, SamSung S23

Models

Model:NexaAIDev/OmniVLM-968M

Problem description & steps to reproduce

load time ,encode image too long

First Bad Commit

Image

Relevant log output

no log

perp avatar Feb 14 '25 04:02 perp