Damien

+1

+1

This is about running LLMs locally on Apple Silicone. Core ML is a framework that can redistribute workload across CPU, GPU & Nural Engine (ANE). ANE is available on all...

This is about running LLMs locally on Apple Silicone. Core ML is a framework that can redistribute workload across CPU, GPU & Nural Engine (ANE). ANE is available on all...

Work in progress on CoreML implementation for [whisper.cpp]. They see x3 performance improvements for some models. (https://github.com/ggerganov/whisper.cpp/discussions/548) you might be interested in. You might also be interested in another implementation...

This is about running LLMs locally on Apple Silicone. Core ML is a framework that can redistribute workload across CPU, GPU & Nural Engine (ANE). ANE is available on all...

Work in progress on CoreML implementation for [whisper.cpp]. They see x3 performance improvements for some models. (https://github.com/ggerganov/whisper.cpp/discussions/548) you might be interested in.

You might also be interested in another implementation [Swift Transformers](https://huggingface.co/blog/swift-coreml-llm). Example of CoreML application https://github.com/huggingface/swift-chat

Please consider adding Core ML model package format support to utilize Apple Silicone Nural Engine + GPU. List of Core ML package format models https://github.com/likedan/Awesome-CoreML-Models

Here is some additional info about running LLMs locally on Apple Silicone. Core ML is a framework that can redistribute workload across CPU, GPU & Nural Engine (ANE). ANE is...