weibo-search icon indicating copy to clipboard operation
weibo-search copied to clipboard

爬取4小时后爬不动并出现struct.error

Open pxaklbe opened this issue 1 year ago • 6 comments

开发者您好,我在爬取数据4小时后发现程序terminal端自动停止,于是用control+c保存断点再爬,但开始遇到各种错误,现在遇到的是Unhandled Error Traceback (most recent call last): File "/Users/cindys/enter/lib/python3.9/site-packages/scrapy/commands/crawl.py", line 31, in run self.crawler_process.start() File "/Users/cindys/enter/lib/python3.9/site-packages/scrapy/crawler.py", line 383, in start reactor.run(installSignalHandlers=False) # blocking call File "/Users/cindys/enter/lib/python3.9/site-packages/twisted/internet/base.py", line 1318, in run self.mainLoop() 后面还有好几行,最后跟着struct.error: unpack requires a buffer of 4 bytes。请问怎样解决呢?

pxaklbe avatar Apr 27 '23 13:04 pxaklbe