ggml icon indicating copy to clipboard operation
ggml copied to clipboard

Published 20 hours ago • verified publisher icon ggerganov

ggml vs onnxruntime on SOC chip

Open Francis235 opened this issue 9 months ago • 1 comments

Hello, I would like to know if anyone has compared the inference of ggml and onnxruntime on SOC in terms of latency, memory usage, %CPU and other indicators? For example, CPU/GPU backend.

Francis235 avatar Apr 30 '24 02:04 Francis235

I am interested in knowing too

bkaruman avatar Jun 06 '24 20:06 bkaruman