500 error calling Ollama API at /api/embeddings

[tcpiplab]

Describe the bug Identical to the bug in #1415 but on Mac OS.

Setup

• Installation npx flowise start
• Flowise Version 1.4.11
• OS: [e.g. macOS, Windows, Linux]
• Browser [e.g. chrome, safari]

Additional context I'm having this exact same problem on Mac OS Sonoma 14.2.1, M2 Pro chip.

time=2024-02-03T14:49:52.666-08:00 level=INFO source=dyn_ext_server.go:156 msg="Starting llama main loop"
[GIN] 2024/02/03 - 14:49:52 | 200 |  4.109615916s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/02/03 - 14:50:00 | 500 |    1.825666ms |       127.0.0.1 | POST     "/api/embeddings"
[GIN] 2024/02/03 - 14:50:04 | 500 |    1.303417ms |       127.0.0.1 | POST     "/api/embeddings"
[GIN] 2024/02/03 - 14:50:22 | 500 |      2.0265ms |       127.0.0.1 | POST     "/api/embeddings"

But it works fine if I do the POST with curl:

$ curl http://localhost:11434/api/embeddings -d '{
  "model": "llama2",
  "prompt": "Here is an article about llamas..."
}'
{"embedding":[1.7122478485107422,0.8994832038879395,......

So I think this must mean that there is something wrong with the body of the POST being sent by the FlowWise Ollama embeddings widget.

Here is an excerpt from the console after starting FlowWise:

$ npx flowise start
2024-02-03 13:56:09 [INFO]: Starting Flowise...
(node:70494) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
2024-02-03 13:56:09 [INFO]: ⚡️ [server]: Flowise Server is listening at 3000
2024-02-03 13:56:09 [INFO]: 📦 [server]: Data Source has been initialized!
2024-02-03 13:56:47 [INFO]: ❌ DELETE /api/v1/chatmessage/1b64af67-db1b-46c5-b886-e8b361002efe?chatId=72da5385-5358-4a31-b5aa-288d46dbece8&chatType=INTERNAL
2024-02-03 13:57:05 [INFO]: ⬆️ POST /api/v1/internal-prediction/1b64af67-db1b-46c5-b886-e8b361002efe
2024-02-03 13:58:29 [ERROR]: Request to Ollama server failed: 500 Internal Server Error
Error: Request to Ollama server failed: 500 Internal Server Error
    at OllamaEmbeddings._request (/opt/homebrew/lib/node_modules/flowise/node_modules/@langchain/community/dist/embeddings/ollama.cjs:101:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async RetryOperation._fn (/opt/homebrew/lib/node_modules/flowise/node_modules/p-retry/index.js:50:12)

Thanks.

[EdwHA]

I had a 500 error with a large text, try to reduce it to a minimum to check the functionality. Good luck! Enable Use Map in the Additional parameters of Ollama Embeddings

[arfeifei]

I had a 500 error with a large text, try to reduce it to a minimum to check the functionality. Good luck! Enable Use Map in the Additional parameters of Ollama Embeddings

Based on your recommendation add a text splitter. then Ollam Embedding goes error-free, Thanks

[VertigoOne1]

I also got it working by reducing the splitter chunk size and enabling use of map. I believe the chunk size reduction alone would have done the trick. The default chunk size is more for OpenAI use or large context models probably. The error from Ollama could actually be better, reviewing their logs does not actually indicate "anything" even at debug.

GIN] 2024/02/14 - 21:47:54 | 500 | 470.419µs | 10.42.0.110 | POST "/api/embeddings"

[ErroneousBosch]

@VertigoOne1 What chunk size did you use?

[wurstache]

I have the same issue, but I tryed with "MMAP" (there is 2 M on the card) and with the same chunk size as with curl still, it don't work. It seems that flowise is spamming ollama with query untill it crashes. So it works with small test texts but doesn't with full document. Maybe partial commit can free the limited memory on my computer.

[wurstache]

@VertigoOne1 What chunk size did you use?

For me too small and I have too much query and to big and Ollama get stuck. about 800 seems good.

[VertigoOne1]

i agree with @wurstache at this point, it seems it spams the api, rather than chunk size having an effect. I've moved to Unstructured API and ollama with llama2 and i have not had an issue since. @ErroneousBosch

The flow is basically unstructured, postgresql vector store, ollama llama2 embeddings, and ollama chat with llama2.

[ErroneousBosch]

I got it working fine with the nomic-embed-text model (which is also lightning fast) by making sure "use MMAP" was on and that I was leveraging my GPU (it had stopped working due to a change in my hosting).

[SikamikanikoBG]

I got it working fine with the nomic-embed-text model (which is also lightning fast) by making sure "use MMAP" was on and that I was leveraging my GPU (it had stopped working due to a change in my hosting).

Saved my day! THANKS! Been dealing with 500 error for 2 weeks now!

[elhakimz]

still got this, when using OllamaEmbeddings (MMAP = on)

2024-05-07 07:46:59 [ERROR]: [server]: Error: Error: Error: Request to Ollama server failed: 500 Internal Server Error Error: Error: Error: Request to Ollama server failed: 500 Internal Server Error at buildFlow (C:\Users\elhak\AppData\Roaming\npm\node_modules\flowise\dist\utils\index.js:453:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async upsertVector (C:\Users\elhak\AppData\Roaming\npm\node_modules\flowise\dist\utils\upsertVector.js:117:32) at async Object.upsertVectorMiddleware (C:\Users\elhak\AppData\Roaming\npm\node_modules\flowise\dist\services\vectors\index.js:9:16) at async createInternalUpsert (C:\Users\elhak\AppData\Roaming\npm\node_modules\flowise\dist\controllers\vectors\inde

[EthraZa]

Same here on Flowise 1.8.1 and Ollama 0.1.41 both running local on Pop!_OS 22.04 LTS with Flowise Docs QnA, changing OpenAI for ChatOllama.

2024-06-10 17:17:08 [INFO]: ⬆️ POST /api/v1/internal-prediction/0b385e7f-aacf-4aa6-8631-53f5b72b0a2e
2024-06-10 17:18:47 [ERROR]: [server]: Error: Request to Ollama server failed: 500 Internal Server Error
Error: Request to Ollama server failed: 500 Internal Server Error
    at OllamaEmbeddings._request (/home/allan/.local/share/fnm/node-versions/v20.10.0/installation/lib/node_modules/flowise/node_modules/@langchain/community/dist/embeddings/ollama.cjs:124:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async RetryOperation._fn (/home/allan/.local/share/fnm/node-versions/v20.10.0/installation/lib/node_modules/flowise/node_modules/p-retry/index.js:50:12)

UPDATE: Turning on MMap in addidional parameters of Ollama Embeddings make it works. My bad!