Cannot use llamafile models on any input containing HTML '<img src="' text

[leighklotz]

If you are doing text inference and the prompt contains an HTML sequence '<img src="' then main.cpp will force call llava_cli and demand an image.

There is no way to override the behavior, so if your input contains such a string, it cannot be processed with llamafile.

$ lamafile-0.6.2 --mode mistral-7b-instruct-v0.2.Q5_K_M.llamafile  --gpu nvidia -f fail.txt
...
Log start
llamafile-0.6.2: fatal error: --mmproj must also be passed when an --image is specified in cli mode
$ 

According to c008e436 the change was copied from ggerganov/llama.cpp upstream, but it does not appear to be present in current versions of ggerganov/llama.cpp.

Perhaps it is an incomplete merge.

The change from c008e436 is below:

   if (!params.image.empty() ||
        params.prompt.find("<img src=\"") != std::string::npos) {
        return llava_cli(argc, argv, &params);
    }