transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard

Published 20 hours ago • verified publisher icon xenova

Is converting a Gemma 2B quantized compatible with transformers.js/onnx?

Open iamhenry opened this issue 1 year ago • 2 comments

Question

I'm new to dev and wanted to know if converting a gemma 2b using the Optimum converter would work for this model?

iamhenry avatar Sep 18 '24 15:09 iamhenry

The tokenizer is supported, but from a quick search in the source code it seems it's not itself supported. You can try WebLLM or Wllama to run that model? That's what I do.

flatsiedatsie avatar Sep 23 '24 13:09 flatsiedatsie

@flatsiedatsie thx for the insight.

i'm thinking of converting the new qwen 2.5 0.5B model.

That should be supported, right?

update: looks like it's already supported! https://huggingface.co/onnx-community/Qwen2.5-0.5B-Instruct

iamhenry avatar Sep 24 '24 20:09 iamhenry