pyspider icon indicating copy to clipboard operation
pyspider copied to clipboard

can not support python3.7

Open osrbot opened this issue 7 years ago • 13 comments

when i run pyspider . error message: Traceback (most recent call last): File "/usr/local/bin/pyspider", line 7, in from pyspider.run import main File "/usr/local/lib/python3.7/site-packages/pyspider/run.py", line 231 async=True, get_object=False, no_input=False):

mac os 10.13 python3.7

osrbot avatar Jul 10 '18 10:07 osrbot

Change all async to other, change async to aasync here.

sam@mac:/usr/local/lib/python3.7/site-packages/pyspider$ grep -Ri async ./* | grep -v async
Binary file ./__pycache__/run.cpython-37.pyc matches
./fetcher/tornado_fetcher.py:from tornado.curl_httpclient import CurlAsyncHTTPClient
./fetcher/tornado_fetcher.py:from tornado.simple_httpclient import SimpleAsyncHTTPClient
./fetcher/tornado_fetcher.py:class MyCurlAsyncHTTPClient(CurlAsyncHTTPClient):
./fetcher/tornado_fetcher.py:class MySimpleAsyncHTTPClient(SimpleAsyncHTTPClient):
./fetcher/tornado_fetcher.py:            self.http_client = MyCurlAsyncHTTPClient(max_clients=self.poolsize,
./fetcher/tornado_fetcher.py:            self.http_client = tornado.httpclient.HTTPClient(MyCurlAsyncHTTPClient, max_clients=self.poolsize)
Binary file ./fetcher/__pycache__/tornado_fetcher.cpython-37.pyc matches
Binary file ./webui/__pycache__/app.cpython-37.pyc matches
sam@mac:/usr/local/lib/python3.7/site-packages/pyspider$ pwd
/usr/local/lib/python3.7/site-packages/pyspider

aaaaasam avatar Jul 24 '18 01:07 aaaaasam

What’s New In Python 3.7

Backwards incompatible syntax changes:

async and await are now reserved keywords.

look here

aaaaasam avatar Jul 24 '18 01:07 aaaaasam

Have you solved this problem?

ShawnXiee avatar Aug 06 '18 12:08 ShawnXiee

#803 already fixed, just need release a new version.

sdvcrx avatar Aug 20 '18 03:08 sdvcrx

@binux Thanks for releasing a new version to solve this bug.

bigmangos avatar Aug 24 '18 09:08 bigmangos

C:\Users\dell-pc>pyspider C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider\libs\uti ls.py:196: FutureWarning: timeout is not supported on your platform. warnings.warn("timeout is not supported on your platform.", FutureWarning) phantomjs fetcher running on port 25555 [I 181017 20:29:08 run:420] phantomjs exited. Exception in thread Thread-4: Traceback (most recent call last): File "c:\programdata\anaconda3\lib\threading.py", line 917, in _bootstrap_inne r self.run() File "c:\programdata\anaconda3\lib\threading.py", line 865, in run self._target(*self._args, **self.kwargs) File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 535, in invoke return callback(*args, **kwargs) File "c:\programdata\anaconda3\lib\site-packages\click\decorators.py", line 17 , in new_func return f(get_current_context(), *args, **kwargs) File "C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider
run.py", line 236, in fetcher Fetcher = load_cls(None, None, fetcher_cls) File "C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider
run.py", line 48, in load_cls return utils.load_object(value) File "C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider
libs\utils.py", line 369, in load_object module = import(module_name, globals(), locals(), [object_name]) File "C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider
fetcher_init
.py", line 1, in from .tornado_fetcher import Fetcher File "C:\Users\dell-pc\AppData\Roaming\Python\Python37\site-packages\pyspider
fetcher\tornado_fetcher.py", line 89 self.async = async_mode ^ SyntaxError: invalid syntax

[I 181017 20:29:08 result_worker:49] result_worker starting... [I 181017 20:29:12 processor:211] processor starting... [I 181017 20:29:12 scheduler:675] scheduler starting... [I 181017 20:29:12 scheduler:614] in 5m: new:0,success:0,retry:0,failed:0 [I 181017 20:29:13 scheduler:810] scheduler.xmlrpc listening on 127.0.0.1:23333 [I 181017 20:30:12 scheduler:614] in 5m: new:0,success:0,retry:0,failed:0 [I 181017 20:31:13 scheduler:614] in 5m: new:0,success:0,retry:0,failed:0 [I 181017 20:32:13 scheduler:614] in 5m: new:0,success:0,retry:0,failed:0

我是安装Anaconda3后,再pip install pyspider ,也是遇到这个报错 更新了最新版本,运行pyspider报错,不能打开

有谁又遇到么?

lymanland avatar Oct 17 '18 12:10 lymanland

Mac os 使用 pip 安装的,python3.7现在还有这个问题

Harold-the-Axeman avatar Feb 26 '19 10:02 Harold-the-Axeman

修改 C:\Users\你的用户名\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\pyspider\run.py 中的async为其他字符就可以啦

44886 avatar Jul 13 '19 10:07 44886

如果是用 Anaconda3 ,可以考虑将python 降级为 3.6 https://www.jianshu.com/p/618e5ebafb01

wangzhihaiSF avatar Jul 16 '19 09:07 wangzhihaiSF

@binux Thanks for releasing a new version to solve this bug.

i already have this bug !!! what version do you mean?

sadegh-khan avatar Oct 01 '19 09:10 sadegh-khan

1 year and a half have passed and this issue is still hovering... macOS Catalina python 3.7.5 pipenv

gitfourteen avatar Nov 25 '19 08:11 gitfourteen

I solved it.

Change to async_ in all places Use vim.

%s/async/async_/g (Replace all async to async_)

ludwigwittgenstein2 avatar Apr 17 '20 05:04 ludwigwittgenstein2

%s/async/async_mode/g

YongjiangChen avatar Sep 15 '22 19:09 YongjiangChen