frob

What was the full command you used to create gemma-2-2B-it-F16.gguf? What is the current contents of the Modelfile?

How big is gemma2-2-2B-it-F16.gguf? This line: ``` INFO:gguf.gguf_writer:gemma22b-tts/gemma-2-2B-it-F16.gguf: n_tensors = 0, total_size = negligible - metadata only ``` leads me to believe that the file contains no model weights. What...

The warning is because the server is not configured with `OLLAMA_NUM_PARALLEL=1`, but doesn't the operation of the model. Does it always fail on the same images? Can you post an...

The ollama CLI recognizes that an image is attached by detecting filepaths in the prompt. A plain filename will be just a random string: ``` $ ollama run moondream:1.8b-v2-q4_0 >>>...

Calling via the API seems to work as well, so if ollama-python is having issues, it might be a different problem. ``` echo '{"model": "moondream:1.8b-v2-q4_0","messages":[{"role":"user","content":"describe this image","images":["'"$(base64 IMG-20220330-WA0002.jpg)"'"]}],"stream":false}' | curl...

it does help to add instruction to the prompt, try `describe this image: ./IMG-20220330-WA0001.jpg`.

Your python script worked fine for me for both images: ``` $ ./6365.py IMG-20220330-WA0001.jpg The image is a newspaper article in Spanish, titled "JBS Registra Lucro de RS 20,5 bilhos...

I don't think `OLLAMA_MAX_VRAM` is a supported variable in the current code base. It may have been used in the past, but now it just sets the value of `MaxVRAM`...

How are you loading the model? How do you send a prompt?

OK, after the model is loaded, check the logs for a line that says `llm_load_tensors: offloaded 29/81 layers to GPU`. The first number is the number layers offloaded for this...