Gemma 2 support

[ltouati]

Hi Team,

Looks like Gemma 2 is not supported by candle yet

[LaurentMazare]

It isn't yet indeed, a PR to add it would be very welcome :) (I'm unlikely to have the cycles to look at it until next week-end)

[EricLBuehler]

I have an implementation here (EricLBuehler/mistral.rs#490), the conversion process was very easy and is outlined here (EricLBuehler/mistral.rs#486), I will probably port this tomorrow.

[dvx]

@EricLBuehler I know mistral.rs does in-situ quantization (which candle doesn't do afaik), wondering if there's any easy way to handle a quantized version of Gemma 2. I tried a few quantized llama variants, but the architectures seem to be quite different.

[creatorrr]

@LaurentMazare is there any work already underway for this issue? Or would adding @EricLBuehler 's implementation be a good way to go?