Minecraft Environment which is contained in the release version is not adapted by Windows

[RyanLoil]

The Release version somehow put Minecraft (which is a independent branch, not in the main branch) in to it. This "Minecraft Environment" has several packages which are only supported in Linux such as Langchain. If you just use pip install metagpt in Windows, you will get error "No module named pwd" as the package pwd is Unix Specific Service Try to use manual installation rather than pip. And please fix this thank you.

中文版本（Chinese Version）： 发行版本中，不知怎么搞的把Minecraft那个Branch拿进来了。这个Branch里有一些Linux依赖，例如Langchain。直接pip install metagpt会报错"No module named pwd"，建议手动安装。请作者修复这个问题，移除Minecraft环境，或者让Minecraft环境变得同时适配Windows和Linux。

[geekan]

@RyanLoil We should not have any dependencies on langchain (main branch) now. Can you post your specific environment and package version?

[RyanLoil]

Here is the dependencies requirement list from Metagpt 0.7.7 release version from pypi

aiohttp==3.8.4 channels==4.0.0 faiss_cpu==1.7.4 fire==0.4.0 typer==0.9.0 lancedb==0.4.0 langchain==0.0.352 loguru==0.6.0 meilisearch==0.21.0 numpy<1.25.0,>=1.24.3 openai==1.6.0 openpyxl beautifulsoup4==4.12.2 pandas==2.0.3 pydantic==2.5.3 python_docx==0.8.11 PyYAML==6.0.1 setuptools==65.6.3 tenacity==8.2.2 tiktoken==0.5.2 tqdm==4.65.0 anthropic==0.8.1 typing-inspect==0.8.0 libcst==1.0.1 qdrant-client==1.7.0 ta==0.10.2 semantic-kernel==0.4.3.dev0 wrapt==1.15.0 aioredis~=2.0.1 websocket-client==1.6.2 aiofiles==23.2.1 gitpython==3.1.40 zhipuai==2.0.1 rich==13.6.0 nbclient==0.9.0 nbformat==5.9.2 ipython==8.17.2 ipykernel==6.27.0 scikit_learn==1.3.2 typing-extensions==4.9.0 socksio~=1.0.0 gitignore-parser==0.1.9 websockets~=12.0 networkx~=3.2.1 google-generativeai==0.3.2 playwright>=1.26 anytree ipywidgets==8.1.1 Pillow imap_tools==1.5.0

[dev] pylint~=3.0.3 black~=23.3.0 isort~=5.12.0 pre-commit~=3.6.0

[ocr] paddlepaddle==2.4.2 paddleocr>=2.0.1 tabulate==0.9.0

[pyppeteer] pyppeteer>=1.0.2

[search-ddg] duckduckgo-search~=4.1.1

[search-google] google-api-python-client==2.94.0

[selenium] selenium>4 webdriver_manager beautifulsoup4

[test] duckduckgo-search~=4.1.1 webdriver_manager tabulate==0.9.0 selenium>4 beautifulsoup4 google-api-python-client==2.94.0 paddlepaddle==2.4.2 paddleocr>=2.0.1 pytest pytest-asyncio pytest-cov pytest-mock pytest-html pytest-xdist pytest-timeout connexion[uvicorn]~=3.0.5 azure-cognitiveservices-speech~=1.31.0 aioboto3~=11.3.0 chromadb==0.4.14 gradio==3.0.0 grpcio-status==1.48.2 pylint==3.0.3 pybrowsers

As you can see, langchain is still in there. Though main branch doesn't have it anymore. Environment details:

OS: Windows 11 Professional Language: Python 3.9 Environment: Python virtual environment (venv)

[RyanLoil]

Also I highly recommended that please do not use the dependencies versions which are too specific, many frameworks got abandoned due to this issue.

[better629]

@RyanLoil maybe you can try main which removed langchain.

[RyanLoil]

@RyanLoil maybe you can try main which removed langchain.

I have already done that. Just in case I submitted this issue.

[geekan]

Thank you very much for asking this question. Python's package management is still a philosophical issue so far. I have tried many management methods, but at present, each method is not completely ideal, so I still choose the most original and least error-prone method. So far, I haven't seen enough benefit from removing all python package versions. If you want to push for this change, can you give us a reason why we need to change it?

[RyanLoil]

The main reason I asked that issue is that MetaGPT is target on a productive multi-agent platform. As for a platform, it's obviously that it is needed to be compatible. However if you make the dependences version too specific, it may get conflict with other applications. For example, I have an application which just built yesterday, and I'd like to run it with MetaGPT's agent, however it uses the latest version of pandas 2.2.1 with brand new function, but MetaGPT uses the certain version of pandas 2.0.3, then I get trouble though I know that MetaGPT can run in pandas 2.2.1. Perhaps no one mentioning that is because the framework now is only for the AI major developers, not for the public, but soon it will, and there will be many stupid issues that may comes out. You need treat the public as freshman of the framework even just a simple question of packages management. That's Flask and Django did in the old time which by now have their own foundation and stable sponsors. Also, removing the specific version of the dependences required the developers to maintain the framework frequently, which should be a healthy framework's necessary condition. I believe this framework can go much further in this AI era, and I also believe that Deep Wisdom has a dream of it of making it commercialized. So, it needs a clearer dependences and structure with documents in details for the open source community to keep contributing with confidence and convenience, and that is called "the Ecology" which is a key for an enterprise to become unreplaceable( management area knowledge).

[geekan]

flask: https://github.com/pallets/flask/blob/main/pyproject.toml django: https://github.com/django/django/blob/main/setup.cfg fastapi: https://github.com/tiangolo/fastapi/blob/master/requirements.txt pydantic: https://github.com/pydantic/pydantic/blob/main/pyproject.toml

After looking at the practices of several frameworks, it seems that >= and frequently updating dependency versions is the best practice. Your suggestions are very good. If possible, can you help us refresh the dependency packages? In addition, since there are many dependent packages, is there any way we can automate this process as much as possible?

[RyanLoil]

I will reinstall MetaGPT this week to confirm the package dependencies. Could you please send me the core packages that MetaGPT use right now? Core packages: the packages which is not depended by other packages except MetaGPT. For now what I knew the core packages are:

pydanmic pandas openai tqdm scikit_learn fire

And yes, you can take a try on pipreqs , a good tools to output the core packages. As for the version controll, usally a brand new framework will only focus on the latest version of its packages, take the latest version that you use as the right side of ">=", and keep update the dependency by using `pip -upgrade' command. If their is conflict happened in the development, then it is time for the framework to get adaptive and update the dependency version in the "requirements.txt". PS: supporting more LLM is an excellent idea, but it's not necessary to put all the LLM packages in the requirements as not all the users use more than one LLM api.

[geekan]

@better629 could you take a look at this?

[better629]

the llm packages are usually small and easy to install. It's true that not all user use all llms but also most users use more than one api for most community users.

[RyanLoil]

Even if they are using more than one api. It's still unnecessary to install all the api package as they are optional. The principal of framework developing is reduced the dependences as much as possible to prevent dependences conflicts and maintenance cost. The other principal is that keeps all the recommended dependences in your requirements newest and adapted as lower the version as possible. Otherwise you are not developing a framework, you are developing a product. That's the difference between the commercial developing and open-source framework developing. Besides, could you please provide core packages list of the MetaGPT?

[geekan]

aiohttp/typer/loguru/numpy/openai/pydantic/PyYAML/tenacity/tiktoken/aiofiles/gitpython

This is the must-have package that I looked directly into requirements.txt and filtered by eye (no verification)

[RyanLoil]

OK I will start with this to find out the dependencies with the possible lowest version.

[RyanLoil]

@geekan Here is the requirements.txt given by pipreqs:

aiohttp==3.8.6 aioredis==2.0.1 chardet==5.2.0 dashscope==1.14.1 faiss_cpu==1.7.4 fire==0.4.0 gitignore_parser==0.1.9 GitPython==3.1.40 imap_tools==1.5.0 ipython==8.12.3 jieba==0.42.1 joblib==1.3.2 lancedb==0.4.0 libcst==1.0.1 loguru==0.6.0 meilisearch==0.21.0 multiprocess==0.70.16 networkx==3.2.1 numpy==1.24.3 playwright==1.42.0 psutil==5.9.8 python_docx==0.8.11 PyYAML==6.0.1 rank_bm25==0.2.2 regex==2023.12.25 scikit_learn==1.3.2 setuptools==58.1.0 setuptools==65.6.3 tenacity==8.2.3 tiktoken==0.6.0 tqdm==4.66.2 websocket_client==1.6.2 websockets==11.0.3

It removed over 30 packages from the mainbranch requirements.txt.

I am still testing these 33 packages. I believe some of them may still can be removed and also I will try as lower version as possible.