No Docker image and looping response on Ollama and Ubuntu.

[Rishabh-Bajpai]

I am trying to run it on an Ubuntu system with local Ollama installed, but I'm facing three issues:

1. The code is unable to create//pull the Docker image.
2. It is using only the CPU (not the GPU). (resolved by changing the llm)
3. The responses are incorrect and mostly show error messages.

I have an Nvidia RTX 4070 and Docker Desktop installed. All drivers and other peripheral components are working fine. I also tested with Ollama on Windows and experienced the same issues. The agent keeps talking to itself on an endless loop.

[OperationTwelve]

Same issue, I am using Ollama and have the models downloaded the main Gemma2 model and the nomic embedding model... and I get the JSON error where it talks to itself forever and maybe its because I do not have perplexity

[Rishabh-Bajpai]

Update: I have tried several models (including instruct, fine tuned for coding/chat models) for both chat and utility purposes. At times, it utilizes GPU, while at other times it does not. Similarly, it sometimes runs within a docker container, whereas at other instances it does not. On occasion, it starts with the right tool (1 out of 40 prompts), but most of the time it fails to work, either displaying LLM errors or taking the wrong direction, and/or failing to stop.

[pjain]

I'm running a dockerized image on a M4 Pro Mac mini and it has gone into an infinite loop

[frdel]

Not all llms are able to control the system properly. Try a bigger, less quantized model.