pyspider
pyspider copied to clipboard
binux
A Powerful Spider(Web Crawler) System in Python.
* pyspider version: 0.3.10 * Operating system: Ubuntu 18.04 * Start up command: pyspider all / pyspider ### Expected behavior No Error ### Actual behavior I'm getting this error when...
Traceback (most recent call last): File "/usr/local/bin/pyspider", line 11, in load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')() File "/usr/local/lib/python2.7/site-packages/pyspider/run.py", line 754, in main cli() File "/usr/local/lib/python2.7/site-packages/click-3.3-py2.7.egg/click/core.py", line 610, in __call__ return self.main(*args, **kwargs) File...
Hi, I'd like to learn how to use pyspider. I normally work in an Anaconda environment. But if I try to do a "conda install" it can't find it. So,...
[E 190926 19:32:54 base_handler:203] HTTP 599: Connection timed out after 20001 milliseconds Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/pyspider/libs/base_handler.py", line 196, in run_task result = self._run_task(task, response) File "/usr/lib/python2.7/site-packages/pyspider/libs/base_handler.py", line...
* pyspider version: 0.3.10 * Operating system: mac os 10.14.6 * Start up command: pyspider all ### Expected behavior No FETCH_ERROR, No pause ### Actual behavior The status become "Pause"....
Dear all, My script was running smoothly for several days. Recently, pyspider only has on_cronjob task and didn't trigger a new crawl job. I've check all the tasks in this...
根据官方文档采用docker-compose搭建了分布式的pyspider。 建立了一个`utils`工程打算将一些常用的函数放进去，比如generate_id()。最初建立`utils`工程的时候，在`generate_id()`函数中只有一句测试用的`return "Hello"`。后来在`generate_id()`函数中加入了代码之后，在别的模块中调用generate_id()函数，无论任何情况只返回"Hello"。在`utils`中加入新的函数，这些新的函数在别的模块中调用时也显示`module 'projects.utils' has no attribute 'xxx'`。 如果在utils中调用自己的函数比如`generate_id()`则没有问题。 刚才试了一下，重启整个集群能够暂时解决问题。但每次更新代码就重启实在不太方便。
Webui load error caused by [deprecated](https://github.com/mar10/wsgidav/blob/master/wsgidav/wsgidav_app.py#L95) use of **domaincontroller** ``` /usr/local/lib/python2.7/dist-packages/amqp/connection.py:292: AMQPDeprecationWarning: The .frame_writer attribute on the connection was accessed before the connection was established. This is supported for now,...
