xenova
Gemma3n browser support
Feature request
As mentioned in https://github.com/huggingface/transformers.js/pull/1348 , gemma3n is currently not supported in web browser, so please bring web browser support.
Motivation
It is important because gemma3n is a very good model, both with text-only modality or with multimodality, would be nice to be able to run it client-side
Your contribution
None. I tried converting gemma3n-2b to ONNX using the script in this repo, thinking that I might be able to contribute something, but the conversion failed.
@xenova onnxruntime is adding 2-bit quantization support for av2, avx512 and neon. Also, recent emscripten versions support avx2 intrinsics. If these 2 are combined to build onnxruntime with 2-bit quantized models inference support for wasm runtime, maybe gemma3n can be run in browser.
Working on it! :)
Hi @xenova Is there any update on WebGPU support for Gemma 3n ?
