mlx-swift-examples icon indicating copy to clipboard operation
mlx-swift-examples copied to clipboard

LLM cannot run on the new neural accelerator computing unit added to the A19 Pro GPU

Open GaoKaipeng2023 opened this issue 1 month ago • 6 comments

While running some third-party LLMs through this project, I noticed that the GPU counters showed that they were not utilizing the new neural accelerator computing units added to the A19 Pro GPU; instead, they were using the general-purpose ALUs. Is there a need for adaptation here? Does anyone have any good ideas? I believe many large models in the future will want to leverage this computing power.

GaoKaipeng2023 avatar Oct 25 '25 03:10 GaoKaipeng2023