MediaCrawler
MediaCrawler copied to clipboard
macos启动程序失败
提示缺少模块tortoise
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 5, in
但是确定已经安装 (venv) userdeMacBook:MediaCrawler user$ pip list Package Version
aerich 0.7.2 aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise 0.1.1 tortoise-orm 0.20.0 typing_extensions 4.10.0
(venv) userdeMacBook:MediaCrawler user$ which python /Users/user/MediaCrawler/venv/bin/python (venv) userdeMacBook:MediaCrawler user$ python --version Python 3.8.9
(venv) userdeMacBook:MediaCrawler user$ pip show tortoise Name: tortoise Version: 0.1.1 Summary: Turtle graphics on the web. Home-page: https://github.com/epeios-q37/tortoise-python Author: Claude SIMON Author-email: License: UNKNOWN Location: /Users/user/MediaCrawler/venv/lib/python3.8/site-packages Requires: atlastk Required-by:
(venv) userdeMacBook:MediaCrawler user$ pip show tortoise-orm Name: tortoise-orm Version: 0.20.0 Summary: Easy async ORM for python, built with relations in mind Home-page: https://github.com/tortoise/tortoise-orm Author: Andrey Bondar Author-email: [email protected] License: Apache-2.0 Location: /Users/user/MediaCrawler/venv/lib/python3.8/site-packages Requires: aiosqlite, iso8601, pypika-tortoise, pytz Required-by: aerich
pip uninstall aerich pip uninstall tortoise pip install tortoise-orm==0.20.0
按照您的指引卸载安装后,还是不行。
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 5, in
aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 (venv) userdeMacBook:MediaCrawler user$
pip uninstall aerich pip uninstall tortoise pip install tortoise-orm==0.20.0
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 5, in
aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 (venv) userdeMacBook:MediaCrawler user$
关掉命令行后,重新启动,仍然还是不行
userdeMacBook:~ user$ cd MediaCrawler
userdeMacBook:MediaCrawler user$ python -m venv venv
userdeMacBook:MediaCrawler user$ source venv/bin/activate
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 5, in
aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0
那把main.py中的db引入先删除了吧,存储用json或csv,这个tortoise-orm 一直都有点版本兼容问题,后面考虑将它完全移除了 https://github.com/NanmiCoder/MediaCrawler/blob/67443737f58b17c355d1b8a24edd2d9507ecb9c4/main.py#L43
不好意思,把main.py和db_config.py中的db引入先删除了
反馈playwright缺失,但是这个确定应该是装过playwright 1.33.0
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 7, in
aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0
我觉得还是你的环境没弄好,或者虚拟环境没生效。
有ide环境吗,尝试在ide打开本项目,使用ide新建虚拟环境试试。
ide里面还是不行,提示找不到playwright 我在ide里面看绝对路径:/Users/user/MediaCrawler/venv/lib/python3.8/site-packages/playwright (venv) userdeMacBook:MediaCrawler user$ pip list Package Version
aerich 0.7.2 aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 23.2.1 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 68.2.0 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 wheel 0.41.2
[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search
Traceback (most recent call last):
File "main.py", line 7, in