Daniel Bevenius

This sounds interesting. Could some more details around this idea be provided?

Hope it is alright that I created this issue without any actual tasks. I'm not sure what needs to be done or how these shims are created but hopefully others...

I get the same using output using the alpaca model as well. Note sure if this helps but I can get this working with the Llama-2-7B-Chat-ggml model following the steps...

I think this is incorrect and that it also might be addressed by https://github.com/ggml-org/llama.cpp/pull/11900, so I'm closing this to avoid confusion.

@alex-jw-brooks I wanted to try this out and made an attempt to [convert](https://github.com/danbev/learning-ai/blob/main/notes/vision/granite.md#convert-granite-vision-model-to-gguf) the model. But I'm not sure if made a mistake in one of the steps there. If...

> Could you please clarify what the issue that you are running into on your branch is? Is it mostly that the model is slow for inference, or did I...

> If the model is running correctly, you should see thousands of tokens per image. I've added some detailed docs for how to convert the model [here](https://github.com/alex-jw-brooks/llama.cpp/blob/granite_vision_docs/examples/llava/README-granitevision.md) - if it...

> now sure how to fix it as i can't find the class ggml_metal_heap_ptr anywhere in either framework This is an issue that we need to look into. The problem...

Thank you for the PR! There is currently work in progress to introduce a new [vision api](https://github.com/ggerganov/llama.cpp/pull/11292), and along side this work there has been work on supporting [mllama](https://github.com/danbev/llama.cpp/tree/vision_2_mllama_support/examples/vision-mllama#llamacppexamplevision-mllama) (Llama...

I think this issue might be completed since the following PRs have been merged: - whisper.android: https://github.com/ggerganov/whisper.cpp/commit/996581c5e2cc5af24496fbc5c0e73703d3bd536f - whisper.objc: https://github.com/ggerganov/whisper.cpp/commit/83b14c357c71b68cd489fb7b555767eec1a30a01 - whisper.swiftui: https://github.com/ggerganov/whisper.cpp/commit/5bb1d58c6ad0838b26e45215dc3c6d3f8733b0a0 There might be other commits related to...