Add role to reflection with llm
Use 'role' in summary_args to use as the role string of the prompt message used to summarize.
Why are these changes needed?
When creating a GroupChat summary, the prompt 'role' is hardcoded as "system". Idea is to enable the initiate_chat() flow to define this by declaring it in system_args. Issue is that there is already a 'role' argument that is passed: (From the docstrings)
"role": role of the message, can be "assistant", "user", "function". This field is only needed to distinguish between "function" or "assistant"/"user".
However, I don't see anywhere in the flow that this value is retrieved or used for anything. I'm not entirely sure what the intention was (potentially to distinguish function calls from conversations, but the flow makes that decision irrespective of the 'role' value), so we can either (1) repurpuse this value to use as the role string to pass in the message dict or (2) add a different argument (called 'summary_role' or something) to avoid potential issues.
Given that, to the best of my knowledge, the old 'role' value wasn't being used, I think the best approach is 1.
With these changes, the argument can still be used to distinguish between function and assistant, but now has a 'default' value of 'system', and is used as the string value for the 'role' attribute in the summary prompt message.
- No doc changes needed
Related issue number
Closes #2492
Checks
- [] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed.
Added a test on test_groupchat.
⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.
Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.
🔎 Detected hardcoded secrets in your pull request
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 10404662 | Triggered | Generic CLI Secret | eff19acf1365e34fe17d9ac0939666f32b3ceda5 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | e973ac38ea4f7b36687ea03aa44f770d7e2ddcac | .github/workflows/dotnet-release.yml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
https://github.com/microsoft/autogen/actions/runs/8881467046/job/24383882709?pr=2527 The code formatting error can be fixed by pre-commit.
https://github.com/microsoft/autogen/actions/runs/8881467028/job/24383887017?pr=2527 The build error is due to the use of "OAI_CONFIG_LIST" in the added test. Could you use a mocked config list for this test which doesn't require real openai keys?
The changes are fine. Please follow @sonichi 's comments. Thanks.
Fixed both comments.
The code formatting error can be fixed with pre-commit.
I installed pre-commit, but since I am not using WSL, I had to uninstall to be able to commit.
Please fix the build error.
@microsoft-github-policy-service agree
Codecov Report
Attention: Patch coverage is 0% with 6 lines in your changes are missing coverage. Please review.
Project coverage is 22.84%. Comparing base (
372ac1e) to head (b23b4ba). Report is 17 commits behind head on main.
| Files | Patch % | Lines |
|---|---|---|
| autogen/agentchat/conversable_agent.py | 0.00% | 6 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #2527 +/- ##
===========================================
- Coverage 33.11% 22.84% -10.27%
===========================================
Files 86 86
Lines 9108 9243 +135
Branches 1938 2131 +193
===========================================
- Hits 3016 2112 -904
- Misses 5837 6888 +1051
+ Partials 255 243 -12
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 22.78% <0.00%> (-10.33%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Please fix the CI error on formatting.
You can install pre-commit so this will be checked before commit. https://microsoft.github.io/autogen/docs/contributor-guide/pre-commit
Not sure how or why, but seems the issue is with the test_groupchat file, which got an update in main that commented all functions and created this issue. This merge from main seems to be the cause this issue was present in this branch: https://github.com/microsoft/autogen/pull/2527/commits/b23b4bafbac51f6e32ff722e9ec4ebeea766b6dc