EasySpider icon indicating copy to clipboard operation
EasySpider copied to clipboard

希望程序可以在发起http请求前判断url是否正常

Open tumuyan opened this issue 2 years ago • 1 comments

错误log打印如下,看上去是在下载url为空的资源,但是我并没有头绪到底哪里有问题。

希望程序可以在发起http请求的时候可以提前判断请求的地址是否正常,如果不正常能跳过这个资源并打印相关信息,这样至少可以避免崩溃。


-------Retrying-------


Exception in thread Thread-4:
Traceback (most recent call last):
  File "easyspider_executestage.py", line 477, in loopExecute
  File "easyspider_executestage.py", line 385, in executeNode
  File "easyspider_executestage.py", line 522, in loopExecute
  File "easyspider_executestage.py", line 371, in executeNode
  File "easyspider_executestage.py", line 969, in getData
  File "easyspider_executestage.py", line 840, in get_content
  File "easyspider_executestage.py", line 48, in download_image
  File "requests\api.py", line 73, in get
  File "requests\api.py", line 59, in request
  File "requests\sessions.py", line 573, in request
  File "requests\sessions.py", line 484, in prepare_request
  File "requests\models.py", line 368, in prepare
  File "requests\models.py", line 439, in prepare_url
requests.exceptions.MissingSchema: Invalid URL '': No scheme supplied. Perhaps you meant http://?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "threading.py", line 1038, in _bootstrap_inner
  File "easyspider_executestage.py", line 167, in run
  File "easyspider_executestage.py", line 362, in executeNode
  File "easyspider_executestage.py", line 385, in executeNode
  File "easyspider_executestage.py", line 499, in loopExecute
  File "easyspider_executestage.py", line 385, in executeNode
  File "easyspider_executestage.py", line 522, in loopExecute
  File "easyspider_executestage.py", line 371, in executeNode
  File "easyspider_executestage.py", line 969, in getData
  File "easyspider_executestage.py", line 840, in get_content
  File "easyspider_executestage.py", line 48, in download_image
  File "requests\api.py", line 73, in get
  File "requests\api.py", line 59, in request
  File "requests\sessions.py", line 573, in request
  File "requests\sessions.py", line 484, in prepare_request
  File "requests\models.py", line 368, in prepare
  File "requests\models.py", line 439, in prepare_url
requests.exceptions.MissingSchema: Invalid URL '': No scheme supplied. Perhaps you meant http://?

Cannot automatically check new version, please use the following command to check whether a new version avaliable and upgrade by pip:
[1;32mpip index versions commandline_config
pip install commandline --upgrade[0m
Thread with task id:  31  is closed

tumuyan avatar Jul 05 '23 13:07 tumuyan

你的任务流程是什么,我不懂你想做什么,可以把你的任务文件发给我看看:[email protected]

NaiboWang avatar Jul 05 '23 14:07 NaiboWang

Bug已修复,下个版本可用。

NaiboWang avatar Jul 05 '23 21:07 NaiboWang