AutoGPT icon indicating copy to clipboard operation
AutoGPT copied to clipboard

DRAFT : Project Abstraction Layer

Open ph-ausseil opened this issue 1 year ago • 17 comments

UNFINISHED do not accept, this is to share advancements

Background

Implementation of ideas regrouped in #3392 for the project level. The implementation should be fully backward compatible. The implementation migrate the current ai_settings.yaml to a project structure but do not change the core , it change the back end.

  • Now = AutoGPT can have 1 agent
  • New back-end = AutoGPT can have 1 project with 1 agent
  • Future = AutoGPT can have N project with 1 agent each
  • Future ++ = AutoGPT can have N project with N agent each

The new back-end is further detailed in REAME.md

  • AIConfig, is an empty class that inherit from AgentModel (for backward compatibility without changing core variable name)
  • AgentModel, is a representation of an Agent and manage CRU of an Agent.
  • Project, is a common goal to various Agent and manage CRU of this project and D of Agent (not yet implemented)
  • ProjectsBroker, create & manage projects

Changes

  • ➕Documentation : Created a projects/readme.md to help user & developper alike to work on the projects
  • ➕Tests : 3 new pytest files
  • ⛔️ AIConfig : Fully removed and kept for backward compatibility so the core is not modified
  • AgentModel : Added
  • Project : Added
  • ProjectsBroker : Added

Changes that affect core :

  • prompt.py a consequence of the abstraction layer was that def construct_full_prompt(ai_config, prompt_generator: Optional[PromptGenerator] = None) -> str: had to be moved out AIConfig
  • ⚡ 1 line in main.py to reflect the change above

Risk :

  • Manage circular imports as the backward compatibility requires imports from Agent to ProjectBroker & the setup is designed to work with imports from ProjectBroker to Agent.

Documentation

  • Every method will be fully doc stringed
  • README.md created
  • Variable created in .env template should be documented if any
  • Any twitch inserted for backward compatibility must have a # TODO and if possible PROJECT-DEPRECATED tag.

Test Plan

  • Unit test with pytest
  • Publication of the branch on Discord & request for testers.

Scenarii :

MVP :

  • [x] Start a first instance of AutoGPT without ai_settings.yaml
  • [ ] Start an instance of AutoGPT with ai_settings.yaml
  • [x] For any of the instance above save & quit after after at least 1 interaction with the agent
  • [ ] Start the instance above and see if the agent can load (type y) & quit after after at least 1 interaction with the agent
  • [ ] Start the instance above and see if we can create a new agent(type n) & quit after after at least 1 interaction with the agent

PR Quality Checklist

  • [x] My pull request is atomic and focuses on a single change.
  • [ ] I have thoroughly tested my changes with multiple different prompts.
  • [x] I have considered potential risks and mitigations for my changes. FULL BACKWARD COMPATIBILITY
  • [x] I have documented my changes clearly and comprehensively.
  • [x] I have not snuck in any "extra" small tweaks changes

Additional informations :

To loop back on features gathered in #3392 :

  • [x] AgentTeam object
  • [x] Dynamic / Extensible Agent (the agent is constructed dynamically allowing flexibility. & plug-ins)
  • [ ] Dynamic / Extensible Project
  • [ ] Dynamic / Extensible Agent Team
  • [ ] Status
  • [ ] State awareness
  • [ ] Implement folder per agent
  • [x] Agent unique ID
  • [ ] Agent budget
  • [ ] Agent budget consumption
  • [ ] Project open_api_key

YAML Option implemented : Option 3

my_project/
├─── team
│          ├── leader : 
│          ├─────────────  agent 1
│          └── specialized_agents : 
│          ├─────────────   agent 2
│          ├─────────────   agent 3
│          ├─────────────  agent 4

Folder Option implemented : Option 1 , could be a MVP

autogpt
├── projects
│      ├── project_1
│      │   ├── settings.yaml
│      ├── project_2
│      │   ├── settings.yaml
│      └── ...

ph-ausseil avatar Apr 29 '23 22:04 ph-ausseil

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 6, 2023 10:39pm

vercel[bot] avatar Apr 29 '23 22:04 vercel[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 29 '23 22:04 github-actions[bot]

Can’t review right now but make sure there’s abstractions to swap the relevant portions of this out when needed

ntindle avatar Apr 29 '23 22:04 ntindle

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 05:04 github-actions[bot]

To show progress : This is a use case for first time use (no ai_settings) https://user-images.githubusercontent.com/303427/235340440-6d48f149-4379-49fa-8a0c-08713b57a8db.mov

ph-ausseil avatar Apr 30 '23 07:04 ph-ausseil

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 10:04 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 13:04 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 18:04 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 19:04 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar Apr 30 '23 22:04 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 01 '23 21:05 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 01 '23 21:05 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 02 '23 17:05 github-actions[bot]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

p-i- avatar May 05 '23 00:05 p-i-

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 06 '23 10:05 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 06 '23 22:05 github-actions[bot]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar May 16 '23 04:05 github-actions[bot]

What's the status of this subproject?

Pwuts avatar May 31 '23 14:05 Pwuts

What's the status of this subproject?

Put this on hold waiting for rearch, as I was waisting time. I could be adapting it soon if it make sense (this has to be assessed)

ph-ausseil avatar May 31 '23 22:05 ph-ausseil

my 2c, working on supporting multiple agents/project is all nice and dandy, but probably should only be worked on once we have at least /some/ form of planning support and a way to support persistent workspaces, so that agents can be safely suspended/resumed (restarted after crashing).

Tackling this any other way will complicate this part of the re-arch IMO

Boostrix avatar Jun 01 '23 12:06 Boostrix

That's why I have put everything on hold , actually there are many things I wanted to see going on and I guess this time is good to look to Three of Thoughs & Hugging Face Transformers Agent.

When I put my things on hold I was going through an idea similar to the one implemented in HF Transformer Agent, while a set of actions are passed to the main agent, created an agent (LeadAgent) who basically inherit from the core agent but manages a set of agents. Also since I saw what HF did with Transformers Agent, I believe there approach with two sets (tools and agent) might be better.

Le jeu. 1 juin 2023 à 14:22, Boostrix @.***> a écrit :

my 2c, working on support multiple agents/project is all nice and dandy, but probably should only be worked on once we have at least /some/ form of planning support and a way to support persistent workspaces, so that agents can be safely suspended/resumed (restarted after crashing).

Tackling this any other way will complicate this part of the re-arch IMO

— Reply to this email directly, view it on GitHub https://github.com/Significant-Gravitas/Auto-GPT/pull/3549#issuecomment-1571956497, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKCQ5WQUS5BBR36P5RUQLXJCCQ5ANCNFSM6AAAAAAXQPGZJM . You are receiving this because you modified the open/close state.Message ID: @.***>

ph-ausseil avatar Jun 01 '23 14:06 ph-ausseil

Some of us have now begun prototyping actual "planning" support based on using the "Tree of Thought" paper and its associated github repo/code - this is now being discussed/integrated as part of the planner plugin, but I suppose it might be better implemented as a core component in the short term ?

Overall, as a project and as a team, we might want to carefully review our current priorities, due to the re-arch, similar projects got quite a bit of time to play catch up with autogpt, and it seems about time to get our act together and carefully prioritize what we're working on and in what order.

Implementing support for multiple agents/projects at this point might be considered premature, and is almost certainly going to complicate efforts to support workspace persistence and proper planning.

Boostrix avatar Jun 01 '23 14:06 Boostrix

Nah I believe you did well, ToT should be part of planning :)

But management of teams may be more related to the command registry .

  • My attempt was : Command Registry is a list of agent some managing tools (such as files)
  • HF implementation : Command registry is split in a list of tool and a list of agent

Advantage of my implementation : Saving character when context is limited with a shorter registry because files operations (move, copy, read, write , delete , ect...) could be managed in an agent. May be simpler

Advantage of HF : May be simpler

Still I like the HF approach as I choose this path because of a technical constraint that should disappear with augmentation of context lengths.

ph-ausseil avatar Jun 01 '23 14:06 ph-ausseil

the re-arch channel does have a dedicated "commands" thread - however, personally, I believe we should use a "namespace" based notation (namespace.tool) - that would be a good way to give more hints to the LLM.

Currently, the LLM tends to loop/cycle unnecessarily, for example when trying to analyze a problem, it keeps calling "analyze_code"

By using a namespace prefix, we could provide additional meta information to the LLM using a well-known syntax

Boostrix avatar Jun 01 '23 16:06 Boostrix

It is relevant but it can be implemented whether this lead agent has a command_registry or an agent_registry, in my work (I'm using my memory) , I created 2 agents for the sake of demonstration "LeadAgent" and FileManagerAgent which names I believe will give you hint to what they can do. There could be ImageManageragent or CoderAgent with specific ML.

There was two reasons to put my job on hold :

  • I was quitting my job and had a lot to do until the end of my contract 31/05 :)
  • I was extending Agent through monkey patches because this has been estimated out of scope since day 1

I believe there are good and bad.

For the good,

I took some distance which is easier to criticize what got done, what could have been done differently. For instance, I guess it is very hard to figure , but projects-related classnames are interesting as they give a semantic meaning to the highest level of interaction, but if we visualize Projects as a tree/arborescences of Agents working toward a common Goal, then a "Project" could be view as a lead agent at the root of the tree, this agent having a specifics loadings functions :)

I actually like the semantic meaning of the class, but it could be non-necessary complexity and I actually haven't made my mind on it. I just know that I would never had the idea if I haven't taken that distance for a month.

For the bad,

I think I could have make most of it in parallel to the rearch have I had few inputs :) , this activity was not at all on the "critical path" and I believe that the "lets do this and considers others things later" approach was a mistake which affected my motivation tbh.

ph-ausseil avatar Jun 01 '23 16:06 ph-ausseil

Also for the sake of argument, I believe that ToT and agent_registry approach may have synergies

ph-ausseil avatar Jun 01 '23 16:06 ph-ausseil

Checked my code, and for I wrote a dummy MidjourneyAgent . I could explain why I believe it’s better to extend an Agent base class with a Midjourney + a Dalle + a foo + a bar class than a generic ImageGeneratorClass.

In this view of Agent Registry, agents are like plug-ins and I can make myself available to argue over different choices when times comes and there is interest.

Le jeudi 1 juin 2023, Pierre-Henri AUSSEIL @.***> a écrit :

It is relevant but it can be implemented whether this lead agent has a command_registry or an agent_registry, in my work (I'm using my memory) , I created 2 agents for the sake of demonstration "LeadAgent" and FileManagerAgent which names I believe will give you hint to what they can do. There could be ImageManageragent or CoderAgent with specific ML.

There was two reasons to put my job on hold :

  • I was quitting my job and had a lot to do until the end of my contract 31/05 :)
  • I was extending Agent through monkey patches because this has been estimated out of scope since day 1

I believe there are good and bad.

For the good,

I took some distance which is easier to criticise what got done, what could have been done differently. For instance, I guess it is very hard to figure , but project-related are interesting as they give a semantic meaning to the highest level of interaction, but if we visualise Projects as a tree/arborescences of agents , then a "Project" could be view as a lead agent at the root of the tree, this agent having a specifics loadings functions :)

I actually like the semanting meaning of the class, but it could be non-necessary complexity and I actually haven't made my mind on it. I just know that I would never had the idea if I haven't taken that distance for a month.

For the bad,

I think I could have make most of it in parallel to the rearch have I had few inputs :) , this activity was not at all on the "critical path" and I believe that the "lets do this and considers others things laters" approach was a mistake which affected my motivation tbh.

Le jeu. 1 juin 2023 à 16:32, Pierre-Henri AUSSEIL @.***> a écrit :

Nah I believe you did well, ToT should be part of planning :)

But gestion of teams may be more related to the command registry .

  • My attempt was : Command Registry is a list of agent somme managing tools (such as files)
  • HF implementation : Commande registry is split in a list of tool and a list of agent

Advantage of my implementation : Saving character when context is limited with a shorter registry because files operations (move, copy, read, write , delete , ect...) could be managed in an agent. May be simpler

Advantage of HF : May be simpler

Still I like the HF approach as I choose this path because of a technical constraint that should disappear with augmentation of context lengths.

Le jeu. 1 juin 2023 à 16:18, Boostrix @.***> a écrit :

Some of us have now begun prototyping actual "planning" support based on using the "Tree of Thought" paper and its associated github repo/code - this is now being discussed/integrated as part of the planner plugin, but I suppose it might be better implemented as a core component in the short term ?

Overall, as a project and as a team, we might want to carefully review our current priorities, due to the re-arch, similar projects got quite a bit of time to play catch up with autogpt, and it seems about time to get our act together and carefully prioritize what we're working on and in what order.

Implementing support for multiple agents/projects at this point might be considered premature, and is almost certainly going to complicate efforts to support workspace persistence and proper planning.

— Reply to this email directly, view it on GitHub https://github.com/Significant-Gravitas/Auto-GPT/pull/3549#issuecomment-1572146807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKCQ6RWK6LOQPSCYJ6N43XJCQFDANCNFSM6AAAAAAXQPGZJM . You are receiving this because you modified the open/close state.Message ID: @.***>

ph-ausseil avatar Jun 01 '23 17:06 ph-ausseil