[Usage] Inconsistent OCR Results with LLaVA 1.6 and Ollama vs. Online Demo

[arcaweb-ch]

Describe the issue

Issue:

I've been testing LLaVA 1.6 with Ollama for OCR tasks and noticed that the online demo at https://llava.hliu.cc consistently outperforms my local tests, despite using identical prompts and parameters. This discrepancy makes me wonder if there's a difference in implementation or configurations between the online demo and the local version I'm using.

Could you provide any insights into this matter or suggest how to achieve parity with the demo's results?

Thanks for your help.

Reference image: example from wikipedia

Prompt:

find the total in the receipt

[wrapss]

ollama uses a non-optimal version of llama.cpp to convert and use llava 1.6, this PR should solve the problem.

[arcaweb-ch]

ollama uses a non-optimal version of llama.cpp to convert and use llava 1.6, this PR should solve the problem.

Thanks, waiting for it.

[ChristianWeyer]

This may be related, as well? https://github.com/haotian-liu/LLaVA/issues/1497