code-assistant icon indicating copy to clipboard operation
code-assistant copied to clipboard

Investigate thread restart techniques

Open phact opened this issue 1 year ago • 0 comments

Today we share the same thread across all llm requests in a session (this translates to as long as the server is up, server restarts, new thread).

Sometimes the llm gets stuck in loops (buggy code in the context) and it struggles to get out of it. Alternatively having history of previous improvements and mistakes could be helpful.

Investigate into when it makes sense to start a new thread or maybe allow the user to control this in the UI.

phact avatar Sep 07 '24 14:09 phact