All-Hands-AI
Multiple LLM working in sync
@mroch @li-boxuan @jeremi @penberg @JensRoland
integrate a feature that can allow user to use multiple llm models in the project with their special expertise
for example :
when user add 3 LLM models into opendevin with specific usage
first LLM should only be use research and browsing like GPT-3.5, Mixtral ,
second LLM model can be used for code generation like GPT-4o, deepseeker , code llama
third LLM model can be used for any reasoning thinking or any other task or role assign by user like GPT-4o, llama3-70b
user can change the model or role anytime in the middle of project or at beginning to get better control of opendevin workspace and it will greatly reduce API cost and increases it productivity and efficiency
If you find this feature request or enhancement useful, make sure to add a 👍 to the issue
Junior dev agent (small code model), senior dev agent (large code model), browser researcher agent (document model), communication agent (psych model). Would like to see the dynamics of at least 4. Does this feels like how LiteLLM and Ollama together can have multiple models running?
We need some sort of refactoring and redesign to support different LLMs for different agents.
I am in general in favor of this idea.
@li-boxuan should we try this it will greatly improve open devin efficiency and easily able to manage task
Supporting different LLMs for different agents is also important for #2363.
Can it be done through the front end using a drop down on a settings page like 'anythingLLM'?
there should be an option that allows users to use the same llm for all the agents or use different llm per agent.
At the moment is it only one dropdown for one single LLM, and even then not every LLM that is on Ollama is there. Would it be slots-based or sets based UX?
I just unassigned myself. I finished the backend part in #2756, and would be great if anyone would like to take the frontend challenge.
Just to clarify, with #2756, and even without any frontend change, you should be able to use different LLMs for different agents. You just need to define it in config.yaml.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
@li-boxuan what do you think the major roadblock is for this? Also have you seen SakanaAI and their advancement?
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
Also, I am not sure how the delegation agent can be used to share work between different worker agents
We have an agent called delegator agent, is that what you mean? You can try it with some simple task, to see how it works: select it in advanced settings, give it a task like "make a script to solve the 4 queens problem, make sure it's tested and documented", and it should delegate to a coder, then a verifier etc.
Note that they're very experimental. That part can use a lot of improvements!
Also, I am not sure how the delegation agent can be used to share work between different worker agents
It's very tricky and hard to get it right. Although delegation mechanism exists, the default agent, CodeActAgent, is a single agent that doesn't delegate.
@enyst very thankful that the delegator agent exists, but imagine a delegator that needs to command things like "pair programming" or socratic dialogue, the moderation between sub-agents are hard to manage, and the "ending" of agents are another issue entirely.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue is stale because it has been open for 40 days with no activity. Remove the stale label or leave a comment, otherwise it will be closed in 10 days.