MediaCrawler icon indicating copy to clipboard operation
MediaCrawler copied to clipboard

macos启动程序失败

Open ywkdw opened this issue 1 year ago • 8 comments

提示缺少模块tortoise (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 5, in import config File "/Users/user/MediaCrawler/config/init.py", line 2, in from .db_config import * File "/Users/user/MediaCrawler/config/db_config.py", line 2, in import db File "/Users/user/MediaCrawler/db.py", line 3, in from tortoise import Tortoise, run_async ModuleNotFoundError: No module named 'tortoise'

但是确定已经安装 (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aerich 0.7.2 aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise 0.1.1 tortoise-orm 0.20.0 typing_extensions 4.10.0

(venv) userdeMacBook:MediaCrawler user$ which python /Users/user/MediaCrawler/venv/bin/python (venv) userdeMacBook:MediaCrawler user$ python --version Python 3.8.9

ywkdw avatar Mar 17 '24 13:03 ywkdw

(venv) userdeMacBook:MediaCrawler user$ pip show tortoise Name: tortoise Version: 0.1.1 Summary: Turtle graphics on the web. Home-page: https://github.com/epeios-q37/tortoise-python Author: Claude SIMON Author-email: License: UNKNOWN Location: /Users/user/MediaCrawler/venv/lib/python3.8/site-packages Requires: atlastk Required-by:

(venv) userdeMacBook:MediaCrawler user$ pip show tortoise-orm Name: tortoise-orm Version: 0.20.0 Summary: Easy async ORM for python, built with relations in mind Home-page: https://github.com/tortoise/tortoise-orm Author: Andrey Bondar Author-email: [email protected] License: Apache-2.0 Location: /Users/user/MediaCrawler/venv/lib/python3.8/site-packages Requires: aiosqlite, iso8601, pypika-tortoise, pytz Required-by: aerich

ywkdw avatar Mar 17 '24 13:03 ywkdw

pip uninstall aerich pip uninstall tortoise pip install tortoise-orm==0.20.0

NanmiCoder avatar Mar 17 '24 13:03 NanmiCoder

按照您的指引卸载安装后,还是不行。 (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 5, in import config File "/Users/user/MediaCrawler/config/init.py", line 2, in from .db_config import * File "/Users/user/MediaCrawler/config/db_config.py", line 2, in import db File "/Users/user/MediaCrawler/db.py", line 3, in from tortoise import Tortoise, run_async ModuleNotFoundError: No module named 'tortoise' (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 (venv) userdeMacBook:MediaCrawler user$

pip uninstall aerich pip uninstall tortoise pip install tortoise-orm==0.20.0

(venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 5, in import config File "/Users/user/MediaCrawler/config/init.py", line 2, in from .db_config import * File "/Users/user/MediaCrawler/config/db_config.py", line 2, in import db File "/Users/user/MediaCrawler/db.py", line 3, in from tortoise import Tortoise, run_async ModuleNotFoundError: No module named 'tortoise' (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 (venv) userdeMacBook:MediaCrawler user$

关掉命令行后,重新启动,仍然还是不行

userdeMacBook:~ user$ cd MediaCrawler userdeMacBook:MediaCrawler user$ python -m venv venv userdeMacBook:MediaCrawler user$ source venv/bin/activate (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 5, in import config File "/Users/user/MediaCrawler/config/init.py", line 2, in from .db_config import * File "/Users/user/MediaCrawler/config/db_config.py", line 2, in import db File "/Users/user/MediaCrawler/db.py", line 3, in from tortoise import Tortoise, run_async ModuleNotFoundError: No module named 'tortoise' (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0

ywkdw avatar Mar 17 '24 14:03 ywkdw

那把main.py中的db引入先删除了吧,存储用json或csv,这个tortoise-orm 一直都有点版本兼容问题,后面考虑将它完全移除了 https://github.com/NanmiCoder/MediaCrawler/blob/67443737f58b17c355d1b8a24edd2d9507ecb9c4/main.py#L43

NanmiCoder avatar Mar 17 '24 14:03 NanmiCoder

不好意思,把main.py和db_config.py中的db引入先删除了 反馈playwright缺失,但是这个确定应该是装过playwright 1.33.0 (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 7, in from base.base_crawler import AbstractCrawler File "/Users/user/MediaCrawler/base/base_crawler.py", line 4, in from playwright.async_api import BrowserContext, BrowserType ModuleNotFoundError: No module named 'playwright' (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 atlastk 0.13.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 24.0 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 49.2.1 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0

ywkdw avatar Mar 17 '24 14:03 ywkdw

我觉得还是你的环境没弄好,或者虚拟环境没生效。

NanmiCoder avatar Mar 17 '24 15:03 NanmiCoder

有ide环境吗,尝试在ide打开本项目,使用ide新建虚拟环境试试。

NanmiCoder avatar Mar 17 '24 15:03 NanmiCoder

ide里面还是不行,提示找不到playwright 我在ide里面看绝对路径:/Users/user/MediaCrawler/venv/lib/python3.8/site-packages/playwright (venv) userdeMacBook:MediaCrawler user$ pip list Package Version


aerich 0.7.2 aiofiles 23.2.1 aiomysql 0.2.0 aiosqlite 0.17.0 annotated-types 0.6.0 anyio 4.3.0 async-timeout 4.0.3 certifi 2024.2.2 click 8.1.7 dictdiffer 0.9.0 exceptiongroup 1.2.0 greenlet 2.0.1 h11 0.14.0 httpcore 0.17.3 httpx 0.24.0 idna 3.6 iso8601 1.1.0 numpy 1.24.4 opencv-python 4.7.0.72 Pillow 9.5.0 pip 23.2.1 playwright 1.33.0 pydantic 2.5.2 pydantic_core 2.14.5 pyee 9.0.4 PyExecJS 1.5.1 PyMySQL 1.1.0 pypika-tortoise 0.1.6 pytz 2024.1 redis 4.6.0 setuptools 68.2.0 six 1.16.0 sniffio 1.3.1 tenacity 8.2.2 tomlkit 0.12.4 tornado 6.3.2 tortoise-orm 0.20.0 typing_extensions 4.10.0 wheel 0.41.2

[notice] A new release of pip is available: 23.2.1 -> 24.0 [notice] To update, run: pip install --upgrade pip (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 7, in from base.base_crawler import AbstractCrawler File "/Users/user/MediaCrawler/base/base_crawler.py", line 4, in from playwright.async_api import BrowserContext, BrowserType ModuleNotFoundError: No module named 'playwright' (venv) userdeMacBook:MediaCrawler user$ python main.py --platform xhs --lt qrcode --type search Traceback (most recent call last): File "main.py", line 7, in from base.base_crawler import AbstractCrawler File "/Users/user/MediaCrawler/base/base_crawler.py", line 4, in from playwright.async_api import BrowserContext, BrowserType ModuleNotFoundError: No module named 'playwright' (venv) userdeMacBook:MediaCrawler user$

ywkdw avatar Mar 18 '24 12:03 ywkdw