autogen
autogen copied to clipboard
microsoft
functionality of manual history cleaning by user proxy added
What is this and how to use it?
Adds ability to clear agents' message history by user proxy manually writing "clear history" within message of user feedback.
Works for a groupchat when one of the agents is user proxy, recommended human_input_mode="ALWAYS".
When user asked to provide manual feedback, he can write "clear history" in his message to remove all the messages from agents history. Writing "clear history <agent_name>" will clear the history of messages only for selected agent. Writing "clear history <nr_of messages_to_preserve>" clears all the history except of last <nr_of messages_to_preserve> messages. Writing "clear history <agent_name> <nr_of messages_to_preserve>" will clear the history for selected agent except last <nr_of messages_to_preserve> messages.
User can noramally write other things in his message. "clear history" string and parameters will cutted out from user reply before passing it to the chat.
Why are these changes needed?
History of messages tends to grow endless very fast during group conversation. According to my experience, many of old messages became useless and just provide noise. Providing unnecesary history to LLM is bad practice because of:
- Model attention distraction. Valuable information could be lost in the mess of noise. Model performance could significantly drop.
- Processing unnecesary history causes huge costs. According to my experience, I'm spending 20-30$ on 3-4h session with Autogen using gpt-4-turbo.
- Also causes long response times.
- There is a risk of context window overflow, especially for models with small/medium context window lenghthes.
That's why having possibility to manage history during conversation is important.
Related issue number
It could be solution or partial solution for #156, but not only.
Checks
- [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed.
Codecov Report
Attention: 5 lines in your changes are missing coverage. Please review.
Comparison is base (
1ab2354) 32.48% compared to head (fe813da) 45.25%.
| Files | Patch % | Lines |
|---|---|---|
| autogen/agentchat/groupchat.py | 92.10% | 1 Missing and 2 partials :warning: |
| autogen/agentchat/conversable_agent.py | 77.77% | 1 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #1230 +/- ##
===========================================
+ Coverage 32.48% 45.25% +12.76%
===========================================
Files 41 41
Lines 4907 4950 +43
Branches 1120 1201 +81
===========================================
+ Hits 1594 2240 +646
+ Misses 3187 2521 -666
- Partials 126 189 +63
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 45.19% <89.36%> (+12.74%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Thanks for the PR! I think it is a great feature to be able to have a concept of "admin" in a group chat. I am wondering if there is a way to move this PR toward that direction? @joshkyh has been working on Graph Group Chat #857 and I am wondering if this concept of admin can be realized through the graph.
Thanks for the PR! I think it is a great feature to be able to have a concept of "admin" in a group chat. I am wondering if there is a way to move this PR toward that direction? @joshkyh has been working on Graph Group Chat #857 and I am wondering if this concept of admin can be realized through the graph.
The concept of "admin" already exists in the current group chat. But it's not used in this PR yet. It should be used if the intention is to only allow one pre-specified admin to clear the history. The current implementation allows any agent to clear the history.
The concept of "admin" already exists in the current group chat. But it's not used in this PR yet. It should be used if the intention is to only allow one pre-specified admin to clear the history.
Ah right. Though I think @GregorD1A1 wants a more structured response message from admin which can be used to perform some pre-defined administrative functions such as clearing histories?
@ekzhu @sonichi There was misleading from my site. For some reason I thought that if "CLEAR HISTORY" in reply.upper(): will be checked only by admin, but in fact as you @sonichi mentioned every agent can clear the history now.
My intent was to allow user proxy (basically regardless if he's admin or not) to clean the history manually. Correcting PR desct-ription.
Although if for some reason we'll find it more useful to allow only admin to clear history, I don't see any problems to implement it that way.
@GregorD1A1 what do you think of @afourney 's comment? Would you like to clean the history in GroupChat.messages too?
@afourney @sonichi yes, it's a good point. I see there aslo messages collected in GroupChat.messages. I need a while to research it and will say later what do I think. Basically, my proposition is to implement clearing GroupChat.messages here if it will be easy thing to do, but if that will something more complicated, I better open separate PR after finishing that one to make PRs smaller and faster-delivirable. What do you think?
@afourney @sonichi ok, I did it. There were problems with list mutability on the way, but managed to solve them.
I had one question in the conversation. Also, could you fix the code format error? You can use pre-commit.
@sonichi what do you think now? Everything is improved, I hope I answered all questions. Can we merge?
@sonichi what do you think now? Everything is improved, I hope I answered all questions. Can we merge?
Thanks. Running final tests.
Seems like a problems with web browser. Have no idea why, changes are totally unrelated. I have no expertise in web browsing functionality, please help.
EDIT: I see it is a known bug with WebSurfer.
Yeah there is a bug in of websurfer's tests. The fix is here, but can't be merged because of another bug in an unrelated notebook code formatting check:
https://github.com/microsoft/autogen/pull/1417
