Stuck in a loop using ollama/llama2

[jab416171]

Describe the bug

when I try to use OpenDevin, it just gets stuck in a loop and doesn't actually do anything.

Setup and configuration

Current version:

a9f469f0e736809699ce3a6c7513de05085ae9d4

My config.toml and environment vars (be sure to redact API keys):

LLM_API_KEY="11111111111111111111"
LLM_BASE_URL="http://localhost:11434"
LLM_MODEL= "ollama/llama2"
LLM_EMBEDDING_MODEL="llama2"
WORKSPACE_DIR="./workspace"

My model and agent (you can see these settings in the UI):

• Model: llama2
• Agent: MonologueAgent

Commands I ran to install and run OpenDevin:

make build
make setup-config
make run

Steps to Reproduce:

1. install ollama and run OpenDevin with the above config

Logs, error messages, and screenshots:

Additional Context

[DGdev91]

Already discussed many times, for example here https://github.com/OpenDevin/OpenDevin/issues/514

Not really OpenDevin's fault, llama2 just isn't smart enough, we need a different model. Mistral 7b works slighly better but still gets easly confused on complex tasks.

I'm not even 100% sure it's really possible to make it work on an open source model right now

[feuler]

Try using another model. I'm currently using "OpenCodeInterpreter-DS-33B" and it worked (more or less). Other options could be "CodeFuse-DeepSeek-33b" or "Phind-CodeLlama-34B-v2" See: https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard

[tonyppe]

hmm ok - we have agentic AI code running with mistral 7B so, it is indeed enough. If it consistently fails here with opendevin then I would be looking into how opendevin is managing the LLM responses to be the correct format as required.

It's also becoming common knowledge than gpt 3.5 is enough to run agentic environments. Even though 3.5 is an argumentative teenager when compared with gpt4,... 3.5 will exceed the capabilities of 4 with good control, retries and examples.

Requiring to run models like 70B is just impossible to work on, in a home lab.

[jab416171]

I switched over to mistral and it doesn't even respond, but it's using my compute resources. I'm trying codellama now, and we're getting a little farther, but still not getting what I asked for, and seeing a bunch of errors. are these known bugs?

[jab416171]

I know you already have #644, but adding a happy path/config.toml for getting started just using ollama to the readme would help a ton.

[stratte89]

Use the oobabooga webui instead, here is a guide https://github.com/OpenDevin/OpenDevin/commit/08a2dfb01af1aec6743f5e4c23507d63980726c0#commitcomment-140559598

Ollama was repeating every sentence like 100 times without doing anything, tried most models, but oobabooge does create files, at the moment it just stops after step 99 but this might get fixed soon, i hope.

[rbren]

Closing in favor of https://github.com/OpenDevin/OpenDevin/issues/326