feat: pass an image as part of the evaluation

[giladgd]

When llama.cpp's support for this will be stable. Hopefully, there will be an official API for this after https://github.com/ggml-org/llama.cpp/pull/11292 is implemented.

[samlhuillier]

Interested in this kind of multimodal support. Any update on progress?

[fozziethebeat]

Does this encompass adding support for running llava models or should that be a separate feature request? I noticed that llama-cpp-python already includes llava support from llama.cpp so this shouldn't be too hard with setting up the bindings.

[giladgd]

I haven't started working on this yet, but it is planned as part of the roadmap. The plan is to add support for llama.cpp's ability to pass an image to a model, which right now only supports LLaVA.

I'll work on this once llama.cpp's API for this is final, to prevent frequent breaking API changes (unlike what happens on some other libraries)

[fozziethebeat]

Make sense. Hopefully llama.cpp finalizes that API.