ollama
ollama copied to clipboard
ollama
Support Mistral's new visual model: Pixtral-12b-240910
Mistral AI just dropped Pixtral, their 12b model with vision support.
- https://github.com/mistralai/mistral-common/releases/tag/v1.4.0
- https://www.reddit.com/r/LocalLLaMA/comments/1fe3x1z/mistral_dropping_a_new_magnet_link/
Related to Pixtral, but more generally regarding Multimodal support in Ollama:
From my experiments today, Ollama is still supporting multi-modal chat with LLaVa (retried today with v0.3.10).
There were indeed some changes in the Llama.cpp server a while back.
I was genuinely interested to understand how Ollama can still handle it while Llama.cpp reportedly cannot anymore. Was Ollama relying on llama-cli, not llama-server?
Turns out it's not relying on either. Ollama is integrating directly with the llama.cpp code base. As for LLaVa support, they lifted the LLaVa support directly from the Llama.cpp server codebase and have been maintaining that in addition to everything else since then...
Ollama team are truly unsung heroes in this technological revolution.
Is it available at Ollama now?
ollama pull pixtral
pulling manifest
Error: pull model manifest: file does not exist
Not yet.
Is it available at Ollama now?
ollama pull pixtral pulling manifest Error: pull model manifest: file does not exist
Not yet.
Same for me
Tried to convert and add this with ollama https://huggingface.co/DewEfresh/pixtral-12b-8bit/tree/main but it seems the architecture is not supported by Ollama (yet).
K:\AI\DewEfresh\pixtral-12b-8bit>ollama create Pixtral-12B-2409 transferring model data 100% converting model Error: unsupported architecture
K:\AI\DewEfresh\pixtral-12b-8bit>ollama create --quantize q8_K_M Pixtral-12B-2409 transferring model data 100% converting model Error: unsupported architecture
I tried to run the Pixtral with python code using an RTX 4060 with 16GB, but it was not possible :(. Perhaps it would work with a 4090 with 24GB.
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 60.00 MiB. GPU 0 has a total capacity of 15.70 GiB of which 34.69 MiB is free. Including non-PyTorch memory, this process has 15.64 GiB memory in use. Of the allocated memory 15.50 GiB is allocated by PyTorch, and 10.20 MiB is reserved by PyTorch but unallocated.
Cant wait to try the gguf version of Pixtral man Ollama gonna deliver a christmas present early for us
I tried to run the Pixtral with python code using an RTX 4060 with 16GB, but it was not possible :(. Perhaps it would work with a 4090 with 24GB.
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 60.00 MiB. GPU 0 has a total capacity of 15.70 GiB of which 34.69 MiB is free. Including non-PyTorch memory, this process has 15.64 GiB memory in use. Of the allocated memory 15.50 GiB is allocated by PyTorch, and 10.20 MiB is reserved by PyTorch but unallocated.
You need to quantize to run an 12b model on 16GB hardware.
please stop spamming here.
multiple people are subscribed to this issue and patiently wait until it is done. all of them receive an email for every "+1" - it is annoying and for sure doesn't speed up anything. thanks for your understanding.
(sorry for another mail, subscribers...)
Anyone else checking the model library a couple times a day waiting for Pixtral, Llama3.2 9B and Molmo-7B to drop? 😄
im not sure if the team is actively working on multimodal support or if they're focusing on something else at the moment. What is certain is that multimodal capabilities will become increasingly essential in the near future, and many users may switch to alternatives that offer this functionality.
On Tue, Oct 1, 2024, 23:11 Robert McDermott @.***> wrote:
Anyone else checking the model library a couple times a day waiting for Pixtral, Llama3.2 9B and Molmo-7B to drop? 😄
— Reply to this email directly, view it on GitHub https://github.com/ollama/ollama/issues/6748#issuecomment-2387083244, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIMKMGFPFQGUUDYTKWQV67TZZMFWZAVCNFSM6AAAAABOANEDJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBXGA4DGMRUGQ . You are receiving this because you commented.Message ID: @.***>
in the near future, and many users may switch to alternatives that offer this functionality.
Not with local models, I guess.
Anyone else checking the model library a couple times a day waiting for Pixtral, Llama3.2 9B and Molmo-7B to drop? 😄
pretty much
Anyone else checking the model library a couple times a day waiting for Pixtral, Llama3.2 9B and Molmo-7B to drop? 😄
pretty much
there is no Llama3.2 9B THERE ARE 1, 3, 11, 90
Anyone else checking the model library a couple times a day waiting for Pixtral, Llama3.2 9B and Molmo-7B to drop? 😄
pretty much
there is no Llama3.2 9B THERE ARE 1, 3, 11, 90
yeah. Thanks. You know what we mean. Anyways: waiting patiently.
please stop spamming here.
multiple people are subscribed to this issue and patiently wait until it is done. all of them receive an email for every "+1" - it is annoying and for sure doesn't speed up anything. thanks for your understanding.
(sorry for another mail, subscribers...)
Please do your Daily checks for "the drop" without spamming. Holy flipping cow.
Just to point out an alternative for now - LM Studio just released with Pixtral support: https://lmstudio.ai/blog/lmstudio-v0.3.5
@oderwat I believe so, yes, since it's enabled by the MLX engine and llama.cpp still doesn't support it. https://github.com/ggerganov/llama.cpp/issues/9440
But I'm sure ollama is going to get Pixtral support very soon, seeing that llama3.2 vision support is being rolled in 0.4
guys I ma save you some time. I learned that pixtral will COME by end of december. top secret. now stop wasting your time checking every few days