ddpasa

@dhiltgen , let me know if you need more tests or additional information.

Hello @dhiltgen > Are you only seeing the performance improvement on multimodal models? I ran inference with the [Phi-2 model here](https://ollama.ai/library/phi) and I think you're right! The v4 version with...

> I'd like to provide some extra data points. > > All my tests have been conducted on Ubuntu 22.04.4, kernel `6.5.0-1016-gcp`. The underneath hardware is a GCP C3 instance...

Looks like there is a bug in llama.cpp, which explains the weird behaviour I was seeing with bakllava: https://github.com/ggerganov/llama.cpp/issues/5545

> Vulkan can also be used on AMD GPUs. I wonder if the official support for Vulkan is being considered. llama.cpp does have official Vulkan support. I was trying to...

> There seems to be an issue with running models that do not entirely fit into VRAM, here is a backtrace of me trying to run dolphin-mixtral with an AMD...

> @ddpasa , isn't there a limit to the amount of vram it can allocate? What if you consider the limit and let Ollama split the model if it does...

Looks like they were added a few days ago, thanks! https://ollama.com/library/llava-llama3 https://ollama.com/library/llava-phi3

Hello @R-N , do you think this is working now? If I try to train a textual inversion embedding with any model other than v1-5-pruned-emaonly, I get crazy RGB artifacts...

Is it possible to make this support non-square images or densepose videos?