autogen
autogen copied to clipboard
microsoft
[Core] Gemini Integration
How to install AutoGen with this PR
Because this PR is still not merged into the main branch, you need to run the following setup to install AutoGen with Gemini features.
pip install https://github.com/microsoft/autogen/archive/gemini.zip
pip install "google-generativeai" "pydash" "pillow" "pydantic==1.10.13"
There are still lots of work to do:
- Multimodal features
- Notebook: more examples, including RAG, competition, etc.
- Robust test cases
- Function calls and other advanced features
- Blog post about the Gemini's integration.
- Token counts
- Gemini's API error handling
Why are these changes needed?
Related issue number
Checks
- [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed.
Codecov Report
Attention: 157 lines in your changes are missing coverage. Please review.
Comparison is base (
00417ed) 34.30% compared to head (43e8f38) 44.66%.
| Files | Patch % | Lines |
|---|---|---|
| autogen/oai/gemini.py | 6.21% | 151 Missing :warning: |
| autogen/token_count_utils.py | 42.85% | 4 Missing :warning: |
| autogen/oai/client.py | 86.66% | 1 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #979 +/- ##
===========================================
+ Coverage 34.30% 44.66% +10.36%
===========================================
Files 42 43 +1
Lines 5110 5279 +169
Branches 1169 1271 +102
===========================================
+ Hits 1753 2358 +605
+ Misses 3213 2721 -492
- Partials 144 200 +56
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 44.61% <14.20%> (+10.34%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Hi BeibinLi,
I am a complete newbie trying to make multi agent teams based on Gemini pro.
I do have the API key. I am trying to install Gemini version of autogen from pip install and it’s installing a different version I think because of some old log file?
I tried to resolve this with gpt-4 and Gemini pro and until now can’t use autogen with Gemini.
Could you guide me through the process of installation autogen with Gemini on Mac m1 or do you maybe have a link to a relevant tutorial?
Thanks to everyone here for the support. It's amazing to experience such advanced tech in this life. You're all awesome!
The terminal below:
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip uninstall pyautogen
Found existing installation: pyautogen 0.2.2
Uninstalling pyautogen-0.2.2:
Would remove:
/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/*
/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/pyautogen-0.2.2.dist-info/*
Proceed (Y/n)? Y
Successfully uninstalled pyautogen-0.2.2
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip install "pyautogen[gemini]~=0.2.0b4"
Collecting pyautogen~=0.2.0b4 (from pyautogen[gemini]~=0.2.0b4)
Using cached pyautogen-0.2.2-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: diskcache in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (5.6.3)
Requirement already satisfied: flaml in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.1)
Requirement already satisfied: openai~=1.3 in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.4.0)
Requirement already satisfied: python-dotenv in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.0)
Requirement already satisfied: termcolor in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.4.0)
Requirement already satisfied: tiktoken in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.5.2)
WARNING: pyautogen 0.2.2 does not provide the extra 'gemini'
Requirement already satisfied: anyio<5,>=3.5.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.2.0)
Requirement already satisfied: distro<2,>=1.7.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.8.0)
Requirement already satisfied: httpx<1,>=0.23.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.25.2)
Requirement already satisfied: pydantic<3,>=1.9.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.5.2)
Requirement already satisfied: sniffio in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.3.0)
Requirement already satisfied: tqdm>4 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.66.1)
Requirement already satisfied: typing-extensions<5,>=4.5 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.9.0)
Requirement already satisfied: NumPy>=1.17.0rc1 in ./venv/lib/python3.10/site-packages (from flaml->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.26.2)
Requirement already satisfied: regex>=2022.1.18 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.10.3)
Requirement already satisfied: requests>=2.26.0 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.31.0)
Requirement already satisfied: idna>=2.8 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.6)
Requirement already satisfied: exceptiongroup>=1.0.2 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.2.0)
Requirement already satisfied: certifi in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.11.17)
Requirement already satisfied: httpcore==1.* in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.2)
Requirement already satisfied: h11<0.15,>=0.13 in ./venv/lib/python3.10/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.14.0)
Requirement already satisfied: annotated-types>=0.4.0 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.6.0)
Requirement already satisfied: pydantic-core==2.14.5 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.14.5)
Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.0)
Using cached pyautogen-0.2.2-py3-none-any.whl (124 kB)
Installing collected packages: pyautogen
Successfully installed pyautogen-0.2.2
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % /Users/marcinniemkiewicz/autogen_test/venv/bin/python /Users/marcinniemkiewicz/autogen_test/t
est.py
Traceback (most recent call last):
File "/Users/marcinniemkiewicz/autogen_test/test.py", line 12, in <module>
from autogen.agentchat.contrib.img_utils import get_image_data, _to_pil
File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/agentchat/contrib/img_utils.py", line 8, in <module>
from PIL import Image
ModuleNotFoundError: No module named 'PIL'
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test %
Hi BeibinLi,
I am a complete newbie trying to make multi agent teams based on Gemini pro.
I do have the API key. I am trying to install Gemini version of autogen from pip install and it’s installing a different version I think because of some old log file?
I tried to resolve this with gpt-4 and Gemini pro and until now can’t use autogen with Gemini.
Could you guide me through the process of installation autogen with Gemini on Mac m1 or do you maybe have a link to a relevant tutorial?
Thanks to everyone here for the support. It's amazing to experience such advanced tech in this life. You're all awesome!
The terminal below:
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip uninstall pyautogen Found existing installation: pyautogen 0.2.2 Uninstalling pyautogen-0.2.2: Would remove: /Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/* /Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/pyautogen-0.2.2.dist-info/* Proceed (Y/n)? Y Successfully uninstalled pyautogen-0.2.2 (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip install "pyautogen[gemini]~=0.2.0b4" Collecting pyautogen~=0.2.0b4 (from pyautogen[gemini]~=0.2.0b4) Using cached pyautogen-0.2.2-py3-none-any.whl.metadata (15 kB) Requirement already satisfied: diskcache in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (5.6.3) Requirement already satisfied: flaml in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.1) Requirement already satisfied: openai~=1.3 in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.4.0) Requirement already satisfied: python-dotenv in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.0) Requirement already satisfied: termcolor in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.4.0) Requirement already satisfied: tiktoken in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.5.2) WARNING: pyautogen 0.2.2 does not provide the extra 'gemini' Requirement already satisfied: anyio<5,>=3.5.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.2.0) Requirement already satisfied: distro<2,>=1.7.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.8.0) Requirement already satisfied: httpx<1,>=0.23.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.25.2) Requirement already satisfied: pydantic<3,>=1.9.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.5.2) Requirement already satisfied: sniffio in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.3.0) Requirement already satisfied: tqdm>4 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.66.1) Requirement already satisfied: typing-extensions<5,>=4.5 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.9.0) Requirement already satisfied: NumPy>=1.17.0rc1 in ./venv/lib/python3.10/site-packages (from flaml->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.26.2) Requirement already satisfied: regex>=2022.1.18 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.10.3) Requirement already satisfied: requests>=2.26.0 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.31.0) Requirement already satisfied: idna>=2.8 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.6) Requirement already satisfied: exceptiongroup>=1.0.2 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.2.0) Requirement already satisfied: certifi in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.11.17) Requirement already satisfied: httpcore==1.* in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.2) Requirement already satisfied: h11<0.15,>=0.13 in ./venv/lib/python3.10/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.14.0) Requirement already satisfied: annotated-types>=0.4.0 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.6.0) Requirement already satisfied: pydantic-core==2.14.5 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.14.5) Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.3.2) Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.0) Using cached pyautogen-0.2.2-py3-none-any.whl (124 kB) Installing collected packages: pyautogen Successfully installed pyautogen-0.2.2 (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % /Users/marcinniemkiewicz/autogen_test/venv/bin/python /Users/marcinniemkiewicz/autogen_test/t est.py Traceback (most recent call last): File "/Users/marcinniemkiewicz/autogen_test/test.py", line 12, in <module> from autogen.agentchat.contrib.img_utils import get_image_data, _to_pil File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/agentchat/contrib/img_utils.py", line 8, in <module> from PIL import Image ModuleNotFoundError: No module named 'PIL' (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test %
Hi @wieniu91 I think if you gonna look at this notebook it will become more clear. https://github.com/microsoft/autogen/blob/1bf1b01c0ca0382eeffe97fc001fab040175390c/notebook/agentchat_gemini.ipynb
Install this way to get latest gemini branch and install libs !pip install https://github.com/microsoft/autogen/archive/gemini.zip !pip install "google-generativeai" "pydash" "pillow"
in my case it didn't work with the latest version of pydantic so I needed to downgrade it in order to work pydantic==1.10.13 But keep in mind that this code seems like still a draft and doesn't completely works yet.
@libraua Thanks for your answer! @wieniu91 I will add more instructions to the notebook. This PR is still under draft, and we will make more changes in the future.
Hi BeibinLi, I am a complete newbie trying to make multi agent teams based on Gemini pro. I do have the API key. I am trying to install Gemini version of autogen from pip install and it’s installing a different version I think because of some old log file? I tried to resolve this with gpt-4 and Gemini pro and until now can’t use autogen with Gemini. Could you guide me through the process of installation autogen with Gemini on Mac m1 or do you maybe have a link to a relevant tutorial? Thanks to everyone here for the support. It's amazing to experience such advanced tech in this life. You're all awesome! The terminal below:
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip uninstall pyautogen Found existing installation: pyautogen 0.2.2 Uninstalling pyautogen-0.2.2: Would remove: /Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/* /Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/pyautogen-0.2.2.dist-info/* Proceed (Y/n)? Y Successfully uninstalled pyautogen-0.2.2 (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % pip install "pyautogen[gemini]~=0.2.0b4" Collecting pyautogen~=0.2.0b4 (from pyautogen[gemini]~=0.2.0b4) Using cached pyautogen-0.2.2-py3-none-any.whl.metadata (15 kB) Requirement already satisfied: diskcache in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (5.6.3) Requirement already satisfied: flaml in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.1) Requirement already satisfied: openai~=1.3 in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.4.0) Requirement already satisfied: python-dotenv in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.0) Requirement already satisfied: termcolor in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.4.0) Requirement already satisfied: tiktoken in ./venv/lib/python3.10/site-packages (from pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.5.2) WARNING: pyautogen 0.2.2 does not provide the extra 'gemini' Requirement already satisfied: anyio<5,>=3.5.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.2.0) Requirement already satisfied: distro<2,>=1.7.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.8.0) Requirement already satisfied: httpx<1,>=0.23.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.25.2) Requirement already satisfied: pydantic<3,>=1.9.0 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.5.2) Requirement already satisfied: sniffio in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.3.0) Requirement already satisfied: tqdm>4 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.66.1) Requirement already satisfied: typing-extensions<5,>=4.5 in ./venv/lib/python3.10/site-packages (from openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (4.9.0) Requirement already satisfied: NumPy>=1.17.0rc1 in ./venv/lib/python3.10/site-packages (from flaml->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.26.2) Requirement already satisfied: regex>=2022.1.18 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.10.3) Requirement already satisfied: requests>=2.26.0 in ./venv/lib/python3.10/site-packages (from tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.31.0) Requirement already satisfied: idna>=2.8 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.6) Requirement already satisfied: exceptiongroup>=1.0.2 in ./venv/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.2.0) Requirement already satisfied: certifi in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2023.11.17) Requirement already satisfied: httpcore==1.* in ./venv/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (1.0.2) Requirement already satisfied: h11<0.15,>=0.13 in ./venv/lib/python3.10/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.14.0) Requirement already satisfied: annotated-types>=0.4.0 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (0.6.0) Requirement already satisfied: pydantic-core==2.14.5 in ./venv/lib/python3.10/site-packages (from pydantic<3,>=1.9.0->openai~=1.3->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.14.5) Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (3.3.2) Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.26.0->tiktoken->pyautogen~=0.2.0b4->pyautogen[gemini]~=0.2.0b4) (2.1.0) Using cached pyautogen-0.2.2-py3-none-any.whl (124 kB) Installing collected packages: pyautogen Successfully installed pyautogen-0.2.2 (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % /Users/marcinniemkiewicz/autogen_test/venv/bin/python /Users/marcinniemkiewicz/autogen_test/t est.py Traceback (most recent call last): File "/Users/marcinniemkiewicz/autogen_test/test.py", line 12, in <module> from autogen.agentchat.contrib.img_utils import get_image_data, _to_pil File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/autogen/agentchat/contrib/img_utils.py", line 8, in <module> from PIL import Image ModuleNotFoundError: No module named 'PIL' (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test %Hi @wieniu91 I think if you gonna look at this notebook it will become more clear. https://github.com/microsoft/autogen/blob/1bf1b01c0ca0382eeffe97fc001fab040175390c/notebook/agentchat_gemini.ipynb
Install this way to get latest gemini branch and install libs !pip install https://github.com/microsoft/autogen/archive/gemini.zip !pip install "google-generativeai" "pydash" "pillow"
in my case it didn't work with the latest version of pydantic so I needed to downgrade it in order to work pydantic==1.10.13 But keep in mind that this code seems like still a draft and doesn't completely works yet.
Thank you! It went forward! Right now it stops thinking the API of gemini pro belongs to OpenAI and is a mistake. Any idea how can I try to solve this?: (venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % /Users/marcinniemkiewicz/autogen_test/venv/bin/pytho n /Users/marcinniemkiewicz/autogen_test/gemini1.py user_proxy (to assistant):
Sort the array with Bubble Sort: [4, 1, 3, 2]
Traceback (most recent call last):
File "/Users/marcinniemkiewicz/autogen_test/gemini1.py", line 62, in
@wieniu91 I believe it happened because you didn't uninstall previous version which you installed by installing older version of autogen.
Try: pip uninstall pyautogen pip install https://github.com/microsoft/autogen/archive/gemini.zip pip install pydantic==1.10.13
@wieniu91 I believe it happened because you didn't uninstall previous version which you installed by installing older version of autogen.
Try: pip uninstall pyautogen pip install https://github.com/microsoft/autogen/archive/gemini.zip pip install pydantic==1.10.13
Hi, thank you! It almost works now! :
(venv) marcinniemkiewicz@MacBook-Pro-Marcin autogen_test % /Users/marcinniemkiewicz/autogen_test/venv/bin/pytho n /Users/marcinniemkiewicz/autogen_test/test.py user_proxy (to assistant):
Sort the array with Bubble Sort: [4, 1, 3, 2]
Traceback (most recent call last): File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 79, in error_remapped_callable return callable_(*args, **kwargs) File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/grpc/_channel.py", line 1160, in call return _end_unary_response_blocking(state, call, False, None) File "/Users/marcinniemkiewicz/autogen_test/venv/lib/python3.10/site-packages/grpc/_channel.py", line 1003, in _end_unary_response_blocking raise _InactiveRpcError(state) # pytype: disable=not-instantiable grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.FAILED_PRECONDITION details = "User location is not supported for the API use." debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2a00:1450:401b:80d::200a%5D:443 {created_time:"2023-12-20T15:41:20.440161+01:00", grpc_status:9, grpc_message:"User location is not supported for the API use."}"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/marcinniemkiewicz/autogen_test/test.py", line 48, in
@wieniu91 https://support.google.com/bard/answer/14294096?hl=en probably your country is not in this list
Hey all with the iminente release of Gemini 1.5, I'm eagerly awaiting this functionality.
Hey all with the iminente release of Gemini 1.5, I'm eagerly awaiting this functionality.
+1
Hi everyone! I've noticed that the Gemini integration is becoming increasingly popular and complex. Therefore, I am planning to pause merging into the "main" branch. Instead, I will request multiple PRs into the "gemini" branch first. Later, after the feature becomes mature, we can start merging into the "main".
I will address all the valuable issues and feedback provided in this PR. Additionally, I will reach out and tag each of you individually in future PRs.
Thanks again for your interest in AutoGen! If you have any questions, please reach out to be via Discord (.beibinli) or Email ([email protected]).