mlx-swift-examples icon indicating copy to clipboard operation
mlx-swift-examples copied to clipboard

Add support for Gemma3 models

Open jolonf opened this issue 7 months ago • 10 comments

I get the following error when loading a Gemma3 model:

unsupportedModelType("gemma3")

jolonf avatar May 02 '25 00:05 jolonf

Yes, #238 needs to be completed first.

davidkoski avatar May 02 '25 01:05 davidkoski

Hey folks! First of all, thank you for mlx-swift, it's great.

Wanted to clarify if there are any plans to support Gemma 3 in mlx-swift and maybe 3n in the nearest future?

tseylerd avatar Jun 07 '25 16:06 tseylerd

There is a PR for it: https://github.com/ml-explore/mlx-swift-examples/pull/238. I think the model is implemented but not fully debugged. If you are interested in it, consider working with the author on this PR! We will get to it eventually but things have been busy.

davidkoski avatar Jun 07 '25 16:06 davidkoski

I hacked together a port of Gemma3n (just the language model) here if its of any use - https://github.com/zsiegel/swift-mlx-gemma3n

I will try to pickup where the others have started and see if I can push anything forward.

zsiegel avatar Jul 05 '25 02:07 zsiegel

I'm currently porting the Gemma3n multimodality from mlx-vlm to mlx-swift.

https://github.com/gradinnovate/mlx-swift-examples/tree/main/Libraries/MLXVLM/Models

gradinnovate avatar Jul 23 '25 07:07 gradinnovate

@gradinnovate see #340 -- you might want to coordinate with @DePasqualeOrg . I think that one is stalled temporarily because the mlx-vlm side was still under development.

davidkoski avatar Jul 23 '25 14:07 davidkoski

Go ahead and build on my draft PR or create your own, since I won't have the capacity to finish this one.

DePasqualeOrg avatar Jul 23 '25 15:07 DePasqualeOrg

Image

current progress....

gradinnovate avatar Jul 27 '25 02:07 gradinnovate

Image Image

now it works for text and image inputs. I will take some time to validate the audio part.

gradinnovate avatar Jul 27 '25 14:07 gradinnovate

Were you able to do the audio part, will be pretty cool!

rudrankriyam avatar Oct 16 '25 18:10 rudrankriyam