AutoGPT icon indicating copy to clipboard operation
AutoGPT copied to clipboard

Feature: Added Self Feedback

Open AbTrax opened this issue 2 years ago • 8 comments

Background

The rationale behind adding the self-feedback feature in the Agent class is to enhance the AI's ability to verify and validate its own thoughts, reasoning, plans, and criticism. This will help improve the AI's decision-making process and allow it to autonomously generate better responses over time. The self-feedback feature is aligned with the project's overall direction of creating an efficient and intelligent AI agent.

Changes

The specific changes made in this pull request involve adding a new method called get_self_feedback() to the Agent class. This method takes in a dictionary of thoughts containing keys such as 'reasoning', 'plan', 'thoughts', and 'criticism'. It combines these elements into a single feedback message and uses the create_chat_completion() function to generate a response based on the input message. This allows the AI to verify and validate its own thoughts before executing a command.

Documentation

The changes are documented through in-code comments and this explanation. The get_self_feedback() method has a docstring that explains its purpose, input, and output. Additionally, the rationale and changes sections above provide further context and explanation.

Test Plan

Testing the self-feedback functionality can be challenging due to the open-ended nature of the responses.

PR Quality Checklist

  • [x] My pull request is atomic and focuses on a single charge.
  • [x] I have thoroughly tested my changes with multiple different prompts.
  • [x] I have considered potential risks and mitigations for my changes.
  • [x] I have documented my changes clearly and comprehensively.
  • [x] I have not snuck in any "extra" small tweaks changes
Due to the open-ended nature of the Self-feedback getting continuous exact answers and negative answers is quite hard whilst getting good open answers is not always the same and can vary differ.

AbTrax avatar Apr 23 '23 13:04 AbTrax

@merwanehamadi Here is the Adverse/Feedback

AbTrax avatar Apr 23 '23 13:04 AbTrax

If it's merged I can add it to the usage.md

AbTrax avatar Apr 23 '23 13:04 AbTrax

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar Apr 23 '23 19:04 github-actions[bot]

If it's merged I can add it to the usage.md

Can you add it in the PR so the docs get updated instantly?

ntindle avatar Apr 23 '23 21:04 ntindle

If it's merged I can add it to the usage.md

Can you add it in the PR so the docs get updated instantly?

You are right I'll do it now

AbTrax avatar Apr 23 '23 23:04 AbTrax

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions[bot] avatar Apr 23 '23 23:04 github-actions[bot]

Codecov Report

Patch coverage: 16.66% and project coverage change: -8.38 :warning:

Comparison is base (29284a5) 49.52% compared to head (139ae35) 41.14%.

:exclamation: Current head 139ae35 differs from pull request most recent head 82db4e9. Consider uploading reports for the commit 82db4e9 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3013      +/-   ##
==========================================
- Coverage   49.52%   41.14%   -8.38%     
==========================================
  Files          64       63       -1     
  Lines        2946     3028      +82     
  Branches      497      498       +1     
==========================================
- Hits         1459     1246     -213     
- Misses       1365     1717     +352     
+ Partials      122       65      -57     
Impacted Files Coverage Δ
autogpt/agent/agent.py 13.86% <16.66%> (-31.27%) :arrow_down:

... and 26 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Apr 24 '23 07:04 codecov[bot]

LGTM - Didn't test though

BillSchumacher avatar Apr 24 '23 07:04 BillSchumacher

Codecov Report

Patch coverage: 16.66% and project coverage change: -8.86 :warning:

Comparison is base (eb0e967) 49.93% compared to head (25b4a15) 41.07%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3013      +/-   ##
==========================================
- Coverage   49.93%   41.07%   -8.86%     
==========================================
  Files          64       64              
  Lines        2900     2924      +24     
  Branches      488      492       +4     
==========================================
- Hits         1448     1201     -247     
- Misses       1333     1667     +334     
+ Partials      119       56      -63     
Impacted Files Coverage Δ
autogpt/agent/agent.py 21.16% <16.66%> (-23.97%) :arrow_down:

... and 14 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Apr 25 '23 00:04 codecov[bot]

Tested locally, LGTM

richbeales avatar Apr 25 '23 05:04 richbeales