mlx-swift-examples
mlx-swift-examples copied to clipboard
LLM cannot run on the new neural accelerator computing unit added to the A19 Pro GPU
While running some third-party LLMs through this project, I noticed that the GPU counters showed that they were not utilizing the new neural accelerator computing units added to the A19 Pro GPU; instead, they were using the general-purpose ALUs. Is there a need for adaptation here? Does anyone have any good ideas? I believe many large models in the future will want to leverage this computing power.