Implement plain speech

[dayofthedave]

Background

The speech feature is very helpful for monitoring Auto-GPT as it chugs through multiple commands in continuous mode or when using the y -N prompt. The current language used for speaking commands is a bit clunky, however, and a more natural choice of words would be nice.

Changes

1. Move CommandRegistry.import_commands logic out of main.py and into CommandRegistry.__init__ so the registry can be accessed by both main.py and app.py
2. Add get_command_speech() function to app.py to generate wording for a command via its description in the registry, along with a few hard-coded phrases for unregistered commands
3. Call get_command_speech from agent.py instead of using default text
4. Refine some command descriptions for use in speech

Documentation

The new get_command_speech function is easy to understand without documentation.

Test Plan

I ran through several different scenarios with speech enabled. Note that I did not test every possible command.

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		May 16, 2023 3:12pm

[anonhostpi]

Man you are going ham on adding TTS/STT support. I saw your other pull just now - #3568

[dayofthedave]

Yeah, I use the speech feature quite a bit to monitor jobs in continuous mode, and it helps to have multiple voices and less robotic language.

On Sat, May 6, 2023 at 11:04 PM Blue Falcon @.***> wrote:

Man you are going ham on adding TTS/STT support. I saw your other pull just now - #3568 https://github.com/Significant-Gravitas/Auto-GPT/pull/3568

— Reply to this email directly, view it on GitHub https://github.com/Significant-Gravitas/Auto-GPT/pull/3885#issuecomment-1537276521, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKYBEELIKI6MKLSSG7IJZ3XE4GKVANCNFSM6AAAAAAXX4DSWA . You are receiving this because you authored the thread.Message ID: @.***>

[k-boikov]

Refine some command descriptions for use in speech

That would change the prompt, please split it to another PR since it would affect a bunch of things.

[dayofthedave]

Sure. Should I pull those changes out of this branch or just close this PR and start over?

[codecov[bot]]

Codecov Report

Patch coverage: 55.26% and project coverage change: +0.78 :tada:

Comparison is base (f424fac) 62.75% compared to head (087b01b) 63.54%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3885      +/-   ##
==========================================
+ Coverage   62.75%   63.54%   +0.78%     
==========================================
  Files          73       73              
  Lines        3367     3396      +29     
  Branches      487      492       +5     
==========================================
+ Hits         2113     2158      +45     
+ Misses       1107     1092      -15     
+ Partials      147      146       -1     

Impacted Files	Coverage Δ
autogpt/commands/audio_text.py	40.00% <ø> (+40.00%)	:arrow_up:
autogpt/commands/git_operations.py	100.00% <ø> (ø)
autogpt/commands/twitter.py	29.41% <ø> (+29.41%)	:arrow_up:
autogpt/commands/web_selenium.py	84.44% <ø> (ø)
autogpt/commands/write_tests.py	60.00% <ø> (+60.00%)	:arrow_up:
autogpt/app.py	37.83% <6.25%> (-5.33%)	:arrow_down:
autogpt/agent/agent.py	57.05% <33.33%> (-0.37%)	:arrow_down:
autogpt/commands/command.py	82.75% <100.00%> (+5.42%)	:arrow_up:
autogpt/commands/execute_code.py	69.56% <100.00%> (ø)
autogpt/commands/file_operations.py	83.13% <100.00%> (ø)
... and 2 more

... and 2 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[k-boikov]

Sure. Should I pull those changes out of this branch or just close this PR and start over?

Yes, you can just remove them an put them in a separate PR.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Pwuts]

Sure. Should I pull those changes out of this branch or just close this PR and start over?

Are you still planning to do this?

[dayofthedave]

Yes, probably not until next week though.

On Thu, Jun 15, 2023 at 6:56 PM Reinier van der Leer < @.***> wrote:

Sure. Should I pull those changes out of this branch or just close this PR and start over?

Are you still planning to do this?

— Reply to this email directly, view it on GitHub https://github.com/Significant-Gravitas/Auto-GPT/pull/3885#issuecomment-1593820967, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKYBEA2XZL6VKMTTFM73OTXLOHKBANCNFSM6AAAAAAXX4DSWA . You are receiving this because you authored the thread.Message ID: @.***>