Gerapy
Gerapy copied to clipboard
Gerapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll...
NotADirectoryError: [Errno 20] Not a directory: '/tmp/AdminSpider-1627912435-5v42z9g9
  定时任务-到期不执行，或配置成功以后，周期性正常执行几天之后，然后距离下次任务数值显示变成负值 就又不正常了
**Describe the bug** 启用异步的TWISTED_REACTOR时候，部署就会报错 **Traceback** Traceback (most recent call last): File "D:\anaconda\envs\scrapy\lib\site-packages\twisted\web\http.py", line 2369, in allContentReceived req.requestReceived(command, path, version) File "D:\anaconda\envs\scrapy\lib\site-packages\twisted\web\http.py", line 1003, in requestReceived self.process() File "D:\anaconda\envs\scrapy\lib\site-packages\twisted\web\server.py", line 229,...
**Describe the bug** DB error executing 'update_job' (database is locked). Retrying with a new DB connection... **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on...
**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll...
ERROR - 2022-05-23 17:35:18,621 - process: 2984 - utils.py - gerapy.server.core.utils - 564 - utils - invalid literal for int() with base 10: 'zw01-teachers-job120' Traceback (most recent call last): File...
**Describe the bug** When using the UI to get a selector the wrong CSS values are in Web CSS Selector compare to the correct values in the xpath selector **To...
使用gerapy管理scrapy-redis，当在一个scrapyd中运行多个scrapy-redis爬虫会报数据库锁住的错误 15/Apr/2022 11:20:07] "GET /api/task/5/status HTTP/1.1" 200 904 [15/Apr/2022 11:20:10] "GET /api/task/5/status HTTP/1.1" 200 904 [15/Apr/2022 11:20:13] "GET /api/task/5/status HTTP/1.1" 200 904 INFO - 2022-04-15 11:20:16,064 - process: 590282 -...
scrapy.item.BaseItem, deprecated in Scrapy 2.2, has now been removed. ([:issue:`5398`](https://github.com/scrapy/scrapy/blob/master/docs/news.rst#id75)) [Twisted Deprecations and Removals](https://github.com/twisted/twisted/blob/trunk/NEWS.rst#deprecations-and-removals-1)
