ollama支持吗

[catzqaz]

[duanshuaimin]

ollama supported is very useful for beginners,thanks

[hexf00]

I'm also in need of this, and I'm confident that integrating with ollama will lead to widespread distribution and usage.

[yunzhichen]

请问在ollama中如何加载int4的2.5版本？目前看到有两个.safetensors文件，是否需要合并？如何合并？非常感谢。 Pls how to load version 2.5 of int4 in ollama? Thank you so much. Currently seeing two .safetensors files, do I need to merge them? How do I merge? Thank you so much.

[yuanjie-ai]

哎 支持下主流的部署哇 ollama

[0x000-0]

ollama只支持GGUF格式的模型进行导入。对于pytorch和safetensors的模型，需要转换为gguf格式之后再导入。 具体步骤，请参考：https://github.com/ollama/ollama/blob/main/docs/import.md

看了下，safetensors格式的要自己转换一下

[0x000-0]

raise NotImplementedError(f'Architecture {arch!r} not supported!') from None

NotImplementedError: Architecture 'MiniCPMV' not supported! 好像转换不了。。

[leeaction]

有gguf文件还不行 还需要编译Ollama的backend llama.cpp

[Cuiunbo]

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is our model in gguf format. https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf @duanshuaimin @leeaction @tyzero @hexf00

[duanshuaimin]

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is our model in gguf format. https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf @duanshuaimin @leeaction @tyzero @hexf00

thanks for your hard work

[yuanjie-ai]

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is our model in gguf format. https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf @duanshuaimin @leeaction @tyzero @hexf00

mmproj-model-f16.gguf这个文件有用吗

[kotaxyz]

why does it hallucinate like that

https://github.com/OpenBMB/MiniCPM-V/assets/105466290/adb052b0-d1e7-4b08-bf7d-298c8f662ff6

[tc-mb]

fork of

Ollama does not directly support every model. I will continue to do an ollama fork this week, so that the community can use ollama run MiniCPMV2.5.

[tc-mb]

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail. and here is our model in gguf format. https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf @duanshuaimin @leeaction @tyzero @hexf00

mmproj-model-f16.gguf这个文件有用吗

有用，这个文件是用于描述多模态模型中的图像部分。

[tc-mb]

why does it hallucinate like that

Video_2024-05-24_044143.mp4

It seems to be because the mmproj-model-f16.gguf is not used, make model loses the input of visual information.

I will make a version of ollama that supports MiniCPMV with instructions for use ASAP.

[duanshuaimin]

create ollama modelfile like :

FROM ./ggml-model-Q4_K_M.gguf TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>" PARAMETER num_keep 24 PARAMETER stop <|start_header_id|> PARAMETER stop <|end_header_id|> PARAMETER stop <|eot_id|>

and execute command :
ollama create xxxx(model name) -f modelfile so you can run ollama locally

doc: https://github.com/ollama/ollama/blob/main/docs/modelfile.md https://github.com/ollama/ollama/blob/main/docs/api.md

[duanshuaimin]

why does it hallucinate like that Video_2024-05-24_044143.mp4

It seems to be because the mmproj-model-f16.gguf is not used, make model loses the input of visual information.

I will make a version of ollama that supports MiniCPMV with instructions for use ASAP.

new version released and ollama supported with priority

[seasoncool]

可否出一個 F16的 gguf格式，Q4的量化 沒有吃滿顯存。

我們嘗試按照readme轉換一了下，發現效果也不是很好，還請指正。

readme

[leeaction]

create ollama modelfile like :

FROM ./ggml-model-Q4_K_M.gguf TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>" PARAMETER num_keep 24 PARAMETER stop <|start_header_id|> PARAMETER stop <|end_header_id|> PARAMETER stop <|eot_id|>

and execute command : ollama create xxxx(model name) -f modelfile so you can run ollama locally

doc: https://github.com/ollama/ollama/blob/main/docs/modelfile.md https://github.com/ollama/ollama/blob/main/docs/api.md

Thank you for you awsome work , I tried this on ollama and meet a issue

I used ollama chat api with images data

The request is：

curl http://127.0.0.1:11434/api/chat -d '{
  "model": "mini-cpm-v2",
  "stream": false,
  "messages": [
    {
      "role": "user",
      "content": "what is in this image?",
      "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAA3VSURBVHgB7Z27r0zdG8fX743i1bi1ikMoFMQloXRpKFFIqI7LH4BEQ+NWIkjQuSWCRIEoULk0gsK1kCBI0IhrQVT7tz/7zZo888yz1r7MnDl7z5xvsjkzs2fP3uu71nNfa7lkAsm7d++Sffv2JbNmzUqcc8m0adOSzZs3Z+/XES4ZckAWJEGWPiCxjsQNLWmQsWjRIpMseaxcuTKpG/7HP27I8P79e7dq1ars/yL4/v27S0ejqwv+cUOGEGGpKHR37tzJCEpHV9tnT58+dXXCJDdECBE2Ojrqjh071hpNECjx4cMHVycM1Uhbv359B2F79+51586daxN/+pyRkRFXKyRDAqxEp4yMlDDzXG1NPnnyJKkThoK0VFd1ELZu3TrzXKxKfW7dMBQ6bcuWLW2v0VlHjx41z717927ba22U9APcw7Nnz1oGEPeL3m3p2mTAYYnFmMOMXybPPXv2bNIPpFZr1NHn4HMw0KRBjg9NuRw95s8PEcz/6DZELQd/09C9QGq5RsmSRybqkwHGjh07OsJSsYYm3ijPpyHzoiacg35MLdDSIS/O1yM778jOTwYUkKNHWUzUWaOsylE00MyI0fcnOwIdjvtNdW/HZwNLGg+sR1kMepSNJXmIwxBZiG8tDTpEZzKg0GItNsosY8USkxDhD0Rinuiko2gfL/RbiD2LZAjU9zKQJj8RDR0vJBR1/Phx9+PHj9Z7REF4nTZkxzX4LCXHrV271qXkBAPGfP/atWvu/PnzHe4C97F48eIsRLZ9+3a3f/9+87dwP1JxaF7/3r17ba+5l4EcaVo0lj3SBq5kGTJSQmLWMjgYNei2GPT1MuMqGTDEFHzeQSP2wi/jGnkmPJ/nhccs44jvDAxpVcxnq0F6eT8h4ni/iIWpR5lPyA6ETkNXoSukvpJAD3AsXLiwpZs49+fPn5ke4j10TqYvegSfn0OnafC+Tv9ooA/JPkgQysqQNBzagXY55nO/oa1F7qvIPWkRL12WRpMWUvpVDYmxAPehxWSe8ZEXL20sadYIozfmNch4QJPAfeJgW3rNsnzphBKNJM2KKODo1rVOMRYik5ETy3ix4qWNI81qAAirizgMIc+yhTytx0JWZuNI03qsrgWlGtwjoS9XwgUhWGyhUaRZZQNNIEwCiXD16tXcAHUs79co0vSD8rrJCIW98pzvxpAWyyo3HYwqS0+H0BjStClcZJT5coMm6D2LOF8TolGJtK9fvyZpyiC5ePFi9nc/oJU4eiEP0jVoAnHa9wyJycITMP78+eMeP37sXrx44d6+fdt6f82aNdkx1pg9e3Zb5W+RSRE+n+VjksQWifvVaTKFhn5O8my63K8Qabdv33b379/PiAP//vuvW7BggZszZ072/+TJk91YgkafPn166zXB1rQHFvouAWHq9z3SEevSUerqCn2/dDCeta2jxYbr69evk4MHDyY7d+7MjhMnTiTPnz9Pfv/+nfQT2ggpO2dMF8cghuoM7Ygj5iWCqRlGFml0QC/ftGmTmzt3rmsaKDsgBSPh0/8yPeLLBihLkOKJc0jp8H8vUzcxIA1k6QJ/c78tWEyj5P3o4u9+jywNPdJi5rAH9x0KHcl4Hg570eQp3+vHXGyrmEeigzQsQsjavXt38ujRo44LQuDDhw+TW7duRS1HGgMxhNXHgflaNTOsHyKvHK5Ijo2jbFjJBQK9YwFd6RVMzfgRBmEfP37suBBm/p49e1qjEP2mwTViNRo0VJWH1deMXcNK08uUjVUu7s/zRaL+oLNxz1bpANco4npUgX4G2eFbpDFyQoQxojBCpEGSytmOH8qrH5Q9vuzD6ofQylkCUmh8DBAr+q8JCyVNtWQIidKQE9wNtLSQnS4jDSsxNHogzFuQBw4cyM61UKVsjfr3ooBkPSqqQHesUPWVtzi9/vQi1T+rJj7WiTz4Pt/l3LxUkr5P2VYZaZ4URpsE+st/dujQoaBBYokbrz/8TJNQYLSonrPS9kUaSkPeZyj1AWSj+d+VBoy1pIWVNed8P0Ll/ee5HdGRhrHhR5GGN0r4LGZBaj8oFDJitBTJzIZgFcmU0Y8ytWMZMzJOaXUSrUs5RxKnrxmbb5YXO9VGUhtpXldhEUogFr3IzIsvlpmdosVcGVGXFWp2oU9kLFL3dEkSz6NHEY1sjSRdIuDFWEhd8KxFqsRi1uM/nz9/zpxnwlESONdg6dKlbsaMGS4EHFHtjFIDHwKOo46l4TxSuxgDzi+rE2jg+BaFruOX4HXa0Nnf1lwAPufZeF8/r6zD97WK2qFnGjBxTw5qNGPxT+5T/r7/7RawFC3j4vTp09koCxkeHjqbHJqArmH5UrFKKksnxrK7FuRIs8STfBZv+luugXZ2pR/pP9Ois4z+TiMzUUkUjD0iEi1fzX8GmXyuxUBRcaUfykV0YZnlJGKQpOiGB76x5GeWkWWJc3mOrK6S7xdND+W5N6XyaRgtWJFe13GkaZnKOsYqGdOVVVbGupsyA/l7emTLHi7vwTdirNEt0qxnzAvBFcnQF16xh/TMpUuXHDowhlA9vQVraQhkudRdzOnK+04ZSP3DUhVSP61YsaLtd/ks7ZgtPcXqPqEafHkdqa84X6aCeL7YWlv6edGFHb+ZFICPlljHhg0bKuk0CSvVznWsotRu433alNdFrqG45ejoaPCaUkWERpLXjzFL2Rpllp7PJU2a/v7Ab8N05/9t27Z16KUqoFGsxnI9EosS2niSYg9SpU6B4JgTrvVW1flt1sT+0ADIJU2maXzcUTraGCRaL1Wp9rUMk16PMom8QhruxzvZIegJjFU7LLCePfS8uaQdPny4jTTL0dbee5mYokQsXTIWNY46kuMbnt8Kmec+LGWtOVIl9cT1rCB0V8WqkjAsRwta93TbwNYoGKsUSChN44lgBNCoHLHzquYKrU6qZ8lolCIN0Rh6cP0Q3U6I6IXILYOQI513hJaSKAorFpuHXJNfVlpRtmYBk1Su1obZr5dnKAO+L10Hrj3WZW+E3qh6IszE37F6EB+68mGpvKm4eb9bFrlzrok7fvr0Kfv727dvWRmdVTJHw0qiiCUSZ6wCK+7XL/AcsgNyL74DQQ730sv78Su7+t/A36MdY0sW5o40ahslXr58aZ5HtZB8GH64m9EmMZ7FpYw4T6QnrZfgenrhFxaSiSGXtPnz57e9TkNZLvTjeqhr734CNtrK41L40sUQckmj1lGKQ0rC37x544r8eNXRpnVE3ZZY7zXo8NomiO0ZUCj2uHz58rbXoZ6gc0uA+F6ZeKS/jhRDUq8MKrTho9fEkihMmhxtBI1DxKFY9XLpVcSkfoi8JGnToZO5sU5aiDQIW716ddt7ZLYtMQlhECdBGXZZMWldY5BHm5xgAroWj4C0hbYkSc/jBmggIrXJWlZM6pSETsEPGqZOndr2uuuR5rF169a2HoHPdurUKZM4CO1WTPqaDaAd+GFGKdIQkxAn9RuEWcTRyN2KSUgiSgF5aWzPTeA/lN5rZubMmR2bE4SIC4nJoltgAV/dVefZm72AtctUCJU2CMJ327hxY9t7EHbkyJFseq+EJSY16RPo3Dkq1kkr7+q0bNmyDuLQcZBEPYmHVdOBiJyIlrRDq41YPWfXOxUysi5fvtyaj+2BpcnsUV/oSoEMOk2CQGlr4ckhBwaetBhjCwH0ZHtJROPJkyc7UjcYLDjmrH7ADTEBXFfOYmB0k9oYBOjJ8b4aOYSe7QkKcYhFlq3QYLQhSidNmtS2RATwy8YOM3EQJsUjKiaWZ+vZToUQgzhkHXudb/PW5YMHD9yZM2faPsMwoc7RciYJXbGuBqJ1UIGKKLv915jsvgtJxCZDubdXr165mzdvtr1Hz5LONA8jrUwKPqsmVesKa49S3Q4WxmRPUEYdTjgiUcfUwLx589ySJUva3oMkP6IYddq6HMS4o55xBJBUeRjzfa4Zdeg56QZ43LhxoyPo7Lf1kNt7oO8wWAbNwaYjIv5lhyS7kRf96dvm5Jah8vfvX3flyhX35cuX6HfzFHOToS1H4BenCaHvO8pr8iDuwoUL7tevX+b5ZdbBair0xkFIlFDlW4ZknEClsp/TzXyAKVOmmHWFVSbDNw1l1+4f90U6IY/q4V27dpnE9bJ+v87QEydjqx/UamVVPRG+mwkNTYN+9tjkwzEx+atCm/X9WvWtDtAb68Wy9LXa1UmvCDDIpPkyOQ5ZwSzJ4jMrvFcr0rSjOUh+GcT4LSg5ugkW1Io0/SCDQBojh0hPlaJdah+tkVYrnTZowP8iq1F1TgMBBauufyB33x1v+NWFYmT5KmppgHC+NkAgbmRkpD3yn9QIseXymoTQFGQmIOKTxiZIWpvAatenVqRVXf2nTrAWMsPnKrMZHz6bJq5jvce6QK8J1cQNgKxlJapMPdZSR64/UivS9NztpkVEdKcrs5alhhWP9NeqlfWopzhZScI6QxseegZRGeg5a8C3Re1Mfl1ScP36ddcUaMuv24iOJtz7sbUjTS4qBvKmstYJoUauiuD3k5qhyr7QdUHMeCgLa1Ear9NquemdXgmum4fvJ6w1lqsuDhNrg1qSpleJK7K3TF0Q2jSd94uSZ60kK1e3qyVpQK6PVWXp2/FC3mp6jBhKKOiY2h3gtUV64TWM6wDETRPLDfSakXmH3w8g9Jlug8ZtTt4kVF0kLUYYmCCtD/DrQ5YhMGbA9L3ucdjh0y8kOHW5gU/VEEmJTcL4Pz/f7mgoAbYkAAAAAElFTkSuQmCC"]
    }
  ]
}'

but I get the response like this：

{"model":"mini-cpm-v2","created_at":"2024-05-24T06:35:43.745508672Z","message":{"role":"assistant","content":"As an AI language model, I cannot see images directly. However, if you provide me with a description of the image or its content, I can try to assist you with your query."},"done_reason":"stop","done":true,"total_duration":12474661490,"load_duration":8495227,"prompt_eval_duration":329466000,"eval_count":39,"eval_duration":12132736000}

The MiniCPM-Llama3-V-2_5-gguf is a Multimodal model, It's seems not work correctly.

[Cuiunbo]

可否出一個 F16的 gguf格式，Q4的量化 沒有吃滿顯存。

我們嘗試按照readme轉換一了下，發現效果也不是很好，還請指正。

readme

It looks like you're using ollama instead of the direct llama.cpp? The current way of ollama should not be able to accept the image features, you can wait for our modification or you are welcome to implement it yourself~!

[seasoncool]

@Cuiunbo thanks for your response, we will test with llama.cpp and reply the result later.

[Cuiunbo]

@seasoncool https://github.com/OpenBMB/llama.cpp Please use our fork, the offical llama.cpp has not merge our PR

[yuanjie-ai]

有没有教程

---原始邮件--- 发件人: @.> 发送时间: 2024年5月25日(周六) 上午10:54 收件人: @.>; 抄送: @.@.>; 主题: Re: [OpenBMB/MiniCPM-V] ollama支持吗 (Issue #97)

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is our model in gguf format. https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf @duanshuaimin @leeaction @tyzero @hexf00

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[Cuiunbo]

https://github.com/OpenBMB/ollama/tree/minicpm-v2.5/examples/minicpm-v2.5 @yuanjie-ai

[chrischjh]

@Cuiunbo Flowing this guide, deployed successfully, but got error when trying to interact with it. Intel chip MAC.

[seasoncool]

@Cuiunbo thanks for your response, we will test with llama.cpp and reply the result later.

I have tested two scenarios multiple times, and the text recognition rate is not very good.

[Cuiunbo]

We notice some reported issues from compromising MiniCPM-Llama3-V 2.5's adaptive visual encoding with Ollama & Llama.cpp's vanilla fixed encoding implementation. We are reimplementing this part for Ollama & Llama.cpp to fully support MiniCPM-Llama3-V 2.5's feature and fix the issue. This update will hopefully be available within a day. Please stay tuned! @seasoncool

[We-IOT]

ollama加载minicpm-v2.5报错： Error: llama runner process has terminated: signal: aborted (core dumped)

重新编译了 下载openBMB fork的ollama和llama.cpp,重新编译后，可以使用了，但是 ollama不能使用以前的模型，另外也pull不下来，总在100%时候出错。

[pxz2016]

go build . 报错 build github.com/ollama/ollama: cannot load cmp: malformed module path "cmp": missing dot in first path element

[pxz2016]

go版本的问题，现在编译成功了。但是运行的时候报错： ./ollama run minicpm-v2.5

你好 Error: an unknown error was encountered while running the model

[pxz2016]

@Cuiunbo Flowing this guide, deployed successfully, but got error when trying to interact with it. Intel chip MAC.

@chrischjh 碰到了同样的问题，你的解决了吗？Ubuntu 20.04.6