Prince Canuma comments

Results 572 comments of


                                            Prince Canuma

Which specific models work with this framework?

Let me know if the issue persists with version 0.1.0

Which specific models work with this framework?

Try ``` pip list | grep mlx ```

Which specific models work with this framework?

Can you try to run this in your terminal ``` python -m mlx_vlm.generate --model mistral-community/pixtral-12b --max-tokens 100 --temp 0.0 --prompt "What animal is this?" ```

Which specific models work with this framework?

Please share the result of ``` pip list | grep mlx ```

Which specific models work with this framework?

Try this model and let me know if the issue persists. `mlx-community/pixtral-12b-8bit`

Which specific models work with this framework?

Something doesn't add up because your logs are saying the model is loading using llava arch instead of pixtral.

Which specific models work with this framework?

I will give it a look.

Which specific models work with this framework?

Found the issue! This version points to llava in the model config. I patched it locally. Don't worry, I will add a condition to fix this at load time. https://huggingface.co/mistral-community/pixtral-12b/blob/main/config.json

Which specific models work with this framework?

> Well this one doesn't just crash out, but it just spins, without producing an answer, either from the command line or via the script above. What are the specs...

Which specific models work with this framework?

Also try the 4bit version instead of the 8bit. `mlx-community/pixtral-12b-4bit`