More control over flow ( interrupt command JIT + choose commands to ignore/stop on + stop when looping )

[eyalk11]

Background

Now it is quite challenging to control the flow. You would need to assume the next N authorised commands are OK, beforehand. If it goes astray, you have nothing to do, but to wait after the batch. This commit make it possible to interrupt just in time and define commands that are more and less relevant with regard to batch.

Changes

1. Added config.commands_to_stop and config.commands_to_ignore.

config.commands_to_stop would stop after the command anyway. For example, you might want to stop on task completion, anyway.

config.commands_to_ignore are commands that authorized automatically and don't increase the counter.

2. Changed spinner class to allow for interruption

• <space> in the middle of thinking means soft-interrupt complete execution and ask for user input after execution).
• <q> means abort - exit task immediately and ask for user input.

Have two implementation for linux and windows.

Introduced a function async_task_and_spin to wait for either task completion or interrupt.

3. Stop if already executed a command

To do so, it keeps all previously executed commands in set.

Documentation

In code

Test Plan

I just checked the code on my own, saw that both soft and hard interrupts working. commands_to_stop and to_ignore seems to be working. There was an error in test_memory_trimmed_from_context that seem unrelated.

    memory_found = memory.get_relevant("Important Information", 5)
    assert memory_found[0] == expected_permanent_memory

Want to see if it fails on workflow.

PR Quality Checklist

• [ ] My pull request is atomic and focuses on a single change. (Arguably, you could split this into adding ignore/to stop and adding spinner. But it is part of whole. )
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

I didn't add tests because of lack of time. A bit of help is wanted, if you think testing is required for it.

This is how soft interrupt looks like. It keeps thinking afterwards.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			May 6, 2023 4:48pm

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Pwuts]

This looks interesting, and like a significant potential UX improvement

[lc0rp]

@eyalk11 We're prepping for release v0.4.3. Please resolve conflicts and stand by as we merge.

[eyalk11]

OK, I am working on this. it is a bit hard to merge. (merge_int branch in my fork).

[netlify[bot]]

❌ Deploy Preview for auto-gpt-docs failed.

Name	Link
🔨 Latest commit	24ad3dd806e107c72efc54962cbdf08db36378c1
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/64ce9132ccb7610008235587

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[eyalk11]

(There are still some merging issues, will solve)

[codecov[bot]]

Codecov Report

Patch coverage: 59.01% and project coverage change: +0.01% :tada:

Comparison is base (3a2d08f) 51.93% compared to head (24ad3dd) 51.94%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3914      +/-   ##
==========================================
+ Coverage   51.93%   51.94%   +0.01%     
==========================================
  Files         117      117              
  Lines        4987     5088     +101     
  Branches      671      694      +23     
==========================================
+ Hits         2590     2643      +53     
- Misses       2199     2235      +36     
- Partials      198      210      +12     

Files Changed	Coverage Δ
autogpt/app/spinner.py	65.62% <55.55%> (-31.75%)	:arrow_down:
autogpt/app/main.py	45.97% <61.70%> (+2.53%)	:arrow_up:
autogpt/config/config.py	79.88% <100.00%> (+0.34%)	:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[eyalk11]

I removed the check for reoccurring commands/ looping (8139493dd98bab7b474cbbf89b24d6f19275d908) as probably prematured. And added comment in https://github.com/Significant-Gravitas/Auto-GPT/issues/3668 . I will open a new PR for it.

Spinner for windows requires keyboard, currently not in requirements (will fail for linux).

[eyalk11]

There is a bug. The bug is that there are leftover executation threads after async_task_and_spin is called because 'pool.shutdown(wait=True)' isn't called. Simply doing.

loop.call_later(0.1, lambda: pool.shutdown(wait=True)) 

in finally won't solve it, because asyncio.run will wait until it is called.

That is why, as long as we don't run natively with asyncio, we need another thread to handle it. Otherwise, we could return a future and execute it opportunistically. My suggested solution is to use another ThreadPoolExecutor that is always available.
or could be like this: https://stackoverflow.com/questions/32059732/send-asyncio-tasks-to-loop-running-in-other-thread

I am open to suggestions. But canceling task without waiting for it to be canceled is problematic.

PS. it seems that short sleep solves it.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[eyalk11]

@Pwuts and @lc0rp I undo refactoring per your request. Consider including this in v0.4.4 . Created branch https://github.com/eyalk11/Auto-GPT/tree/interrupt_commands_with_refactoring that I can PR if you want.

[dschonholtz]

AI-Maintainer Review for PR - More control over flow (interrupt command JIT + choose commands to ignore/stop on + stop when looping)

Title and Description :+1:

The Title and Description are clear, concise, and helpful

The title and description of the pull request are clear and concise. They effectively communicate the purpose of the changes, which is to provide more control over the flow in the Auto-GPT project. The description provides a detailed explanation of the changes and their expected behavior.

Scope of Changes :+1:

The changes are narrowly focused

The changes in the pull request are narrowly focused on addressing a specific issue related to controlling the flow in the Auto-GPT project. The modifications all contribute to the same goal of providing more control over the flow, and it does not appear that the author is trying to resolve multiple unrelated issues simultaneously.

Testing :warning:

The testing approach could be more comprehensive

The author has tested the changes and provided a description of the testing process. However, the testing approach could be more comprehensive. It would be beneficial to include more detailed test cases and potentially automated tests to ensure the new features work as expected in various scenarios.

Documentation :warning:

Some functions, classes, or methods lack docstrings

The following functions, classes, or methods lack docstrings:

• async_task_and_spin in the Agent class
• start_interaction_loop in the Agent class
• _resolve_pathlike_command_args in the Agent class
• spin in the Spinner class
• update_message in the Spinner class

These should be updated to include docstrings describing their behavior, arguments, and return values.

Suggested Changes

• Add docstrings to the functions, classes, or methods mentioned above. The docstrings should describe the behavior, arguments, and return values of each. This will improve code readability and maintainability.
• Consider enhancing the testing approach to include more detailed test cases and potentially automated tests. This will help ensure the new features work as expected in various scenarios and catch any potential issues early.

Potential Issues

• The async_task_and_spin function in the Agent class uses asyncio and threading together, which can lead to complex and hard-to-debug issues. Consider refactoring this to use only asyncio or only threading, if possible.
• The Spinner class now has an interruptable attribute, but it's not clear how this is used if the spinner is not interruptable. Consider adding comments or documentation to clarify this.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[eyalk11]

I did restructuring (undoing refactoring) and merged master.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[eyalk11]

I added test for command_to_stop/ignore.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[lc0rp]

@eyalk11 The agent was updated recently, probably the reason for the conflicts.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.