Error condensing thoughts

[jabbor]

Describe the bug

When creating a simple python program, ('Hello, World!'), opendevin gets into a loop:

Steps to Reproduce

1. run with ollama (mixtral, nomic-embed-text)
2. not with Docker

Expected behavior Not getting in endless loop

Actual behavior

Additional context

[enyst]

Thanks for the report and screenshot, that shows another issue too.

Stopping infinite loops is tracked here https://github.com/OpenDevin/OpenDevin/issues/326

[rbren]

@jabbor what does git log -n1 show? Does the error persist if you pull latest?

Pretty sure we fixed this yesterday

[jabbor]

I'll have a look when I'm back home. I'll let you know.

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Robert Brennan @.> Sent: Monday, April 1, 2024 5:02:15 PM To: OpenDevin/OpenDevin @.> Cc: jabbor @.>; Mention @.> Subject: Re: [OpenDevin/OpenDevin] Error condensing thoughts (Issue #495)

@jabborhttps://github.com/jabbor what does git log -n1 show? Does the error persist if you pull latest?

Pretty sure we fixed this yesterday

— Reply to this email directly, view it on GitHubhttps://github.com/OpenDevin/OpenDevin/issues/495#issuecomment-2029901587, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL5TMOX67KOS5ATLKOB636DY3FZHPAVCNFSM6AAAAABFRMYRLSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZHEYDCNJYG4. You are receiving this because you were mentioned.Message ID: @.***>

[enyst]

@jabbor what does git log -n1 show? Does the error persist if you pull latest?

Pretty sure we fixed this yesterday

I'm not sure about that... The console log displays "Original JSON" and "Repaired JSON", which are from the json_repair merge. So this is after https://github.com/OpenDevin/OpenDevin/commit/4404b9af243d5281ebb3587e2daea23cb20fc281 . 😔

[rbren]

Yeah I see the issue now. For some weird reason the LLM is putting \ characters into the JSON to escape underscores. repair is double-escaping the \ instead of removing it.

[rbren]

@jabbor which model are you using? GPT?

Edit: nvm, I see it's mixtral+ollama now

[rbren]

https://github.com/mangiucugna/json_repair/issues/19

[jabbor]

This is the output of the git log:

I'm going to try another model. Keep you guys posted.

[archenovalis]

instead of repairing json, a better solution would be to never allow the llm to output improper json. https://github.com/outlines-dev/outlines

[rbren]

Agree--I'm starting to lean against the json repair and towards just erroring and telling the LLM more about the error

[rbren]

JSON repair has this fix now. We should work on a more robust long-term solution though