Georgi Gerganov

ggml : add metal backend registry / device

2

target #9707 Adapt the Metal backend to the new registry and device interfaces. - [x] I have read the [contributing guidelines](https://github.com/ggerganov/llama.cpp/blob/master/CONTRIBUTING.md) - Self-reported review complexity: - [ ] Low -...

metal : compile-time kernel args and params

4

I was just thinking about this idea, so writing it down for future research. We should be able to fairly easy generate model-specific Metal code that has hardcoded kernels for...

llama : tool for evaluating quantization results per layer

8

Following up on #2421, I think we should implement some better way to observe at which point of the inference the results start to deviate significantly between the classical and...

changelog : `libllama` API

9

# Overview This is a list of changes to the public interface of the `llama` library. Collaborators are encouraged to edit this post in order to reflect important changes to...

llama : refactor llama_kv_cache, llama_context and llm_build_context

23

## Overview This PR is an intermediate step towards a more generic implementation that will support different underlying implementations of `llama_kv_cache`, `llama_context` and the graph building logic (a.k.a. `llm_build_context`). The...

ggml : add ANE backend

9

According to this https://github.com/ggerganov/llama.cpp/discussions/336#discussioncomment-11184134, there is a new CoreML API and an ANE backend might be possible to implement with latest Apple software/hardware.

llama : create llamax library

18

Depends on: https://github.com/ggerganov/llama.cpp/issues/5214 The `llamax` library will wrap `llama` and expose common high-level functionality. The main goal is to ease the integration of `llama.cpp` into 3rd party projects. Ideally, most...

build : start using a ggml submodule

3

Currently, the [ggml](https://github.com/ggerganov/ggml) source tree is synchronized into the source tree of `whisper.cpp`: https://github.com/ggerganov/whisper.cpp/tree/master/ggml This is not ideal, because the `ggml` source code is duplicated and has to be synchronized...

tests : add WER benchmarks

9

It would be nice to start measuring the word error rate (WER) of `whisper.cpp` across some representative dataset: - short audio - long audio - english - non-english - etc....

wasm : update and fix examples

1

The Emscripten build for the WASM examples hasn't been maintained for a very long time. There are likely regressions that have been introduced that would need to be fixed. Last...