mlx-vlm Add support for Gemma 3?

As far as I can tell all Gemma 3 models are multimodal except maybe 1B ones? Not sure but it says all of them on their HF.

Mar 12 '25 08:03 alexgusevski

Okay saw now that you already support it! Nice!

The 1B one seems to not be able to be converted through this mlx_vlm library, but it is not able to be converted with mlx_lm library either right now.

Mar 12 '25 08:03 alexgusevski

I already made a PR to MLX-LM to support 1B.

https://github.com/ml-explore/mlx-examples/pull/1336

Mar 12 '25 09:03 Blaizzy

I already made a PR to MLX-LM to support 1B.

https://github.com/ml-explore/mlx-examples/pull/1336

You're very quick! Great work!

Would that PR also work for text only use of larger models, like 27B?

Mar 12 '25 09:03 vlbosch

Yes, it will :)

Mar 12 '25 11:03 Blaizzy

Thanks!

Mar 12 '25 11:03 Blaizzy

I just tried the converted model in "--chat"-mode, but as response to a text-only query I get only "< pad >" as output

Mar 12 '25 13:03 vlbosch

@Blaizzy The 12b model does not seem to work for me, it just outputs a lot of nonsense... But, when you do add the compatibility, would it be possible to add fine-tuning?

Mar 13 '25 08:03 psm-2

@psm-2 Same, this is what I get when running the 4B model, got it from mlx community, havent tried other versions of it.

Oh and seems @vlbosch also gets this

Mar 13 '25 12:03 alexgusevski

@psm-2 Same, this is what I get when running the 4B model, got it from mlx community, havent tried other versions of it.
Oh and seems [@vlbosch](https://github.com/vlbosch) also gets this

same on 27B 8bit

Mar 14 '25 05:03 BlackLuny

More people having same issue as above: https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/513

Mar 17 '25 11:03 alexgusevski

This should be fixed!

Mar 24 '25 13:03 FL33TW00D