candle
candle copied to clipboard
Implement the Gemma 3 vision models!
- [x] Add vision part - mmproj
- [x] Implement forward pass
- [x] Add inputs processor (pan & scan, normalize, etc)
- [x] Example
when i run the 4b and 12b like
cargo run --example gemma_vision --release --features cuda,cudnn -- --prompt "describe this image" --which v3_4b --image /home/pro/Desktop/sd_final.png
it downloads the models and then i get error
avx: true, neon: false, simd128: false, f16c: true
temp: 0.00 repeat-penalty: 1.10 repeat-last-n: 64
retrieved the files in 98.089819ms
Error: data did not match any variant of untagged enum Gemma3Config
is this working? its been sitting here for a while would be great to have Gemma vision models in candle especially if the work is done..?