ggml icon indicating copy to clipboard operation
ggml copied to clipboard

[Feature request] Add support/demo implementation for Qwen-VL GGUF model

Open CoruNethron opened this issue 2 years ago • 2 comments

Hello. May be this can be interesting for future roadmap: https://github.com/QwenLM/Qwen-VL

It is multimodal and multilangual 7B model, able to analyze image, including text recognition and compare two (at least) images. Also able to detect bounding box of an object within image.

Seems, like it also beats some very good 13B models in pure textual context.

photo_2023-08-30_04-40-45

Would be nice to see it running quantized in GGUF.

CoruNethron avatar Aug 30 '23 06:08 CoruNethron

Closing this, as Qwen inference has being added few days ago: https://github.com/QwenLM/qwen.cpp

CoruNethron avatar Oct 17 '23 02:10 CoruNethron

Sorry for increasing entropy here, just realized, that recently implemented inference is for another model: https://github.com/QwenLM/Qwen vs https://github.com/QwenLM/Qwen-VL So, I reopen this feature request.

CoruNethron avatar Oct 17 '23 03:10 CoruNethron