sina-weibo-crawler icon indicating copy to clipboard operation
sina-weibo-crawler copied to clipboard

TypeError

Open jyjune24 opened this issue 6 years ago • 6 comments

Traceback (most recent call last): File "C:/Users/June/Desktop/sina-weibo-crawler-master/spider.py", line 12, in print(crawler.crawl(url = 'http://weibo.cn/yaochen')) File "C:\Users\June\Desktop\sina-weibo-crawler-master\wcrawler.py", line 97, in crawl req = self.__get_request(turl) File "C:\Users\June\Desktop\sina-weibo-crawler-master\wcrawler.py", line 257, in __get_request stderr.write(json.dumps(self.data, ensure_ascii=False, sort_keys=True, indent=4).encode('utf-8', 'replace')) TypeError: write() argument must be str, not bytes

请问这是什么问题呢?是由于微博的api限制吗?

jyjune24 avatar Apr 09 '19 03:04 jyjune24

我查了一下,是因为微博从http转向https了,导致url失效了。我已经在提交 https://github.com/intfloat/sina-weibo-crawler/commit/eb6975d94a0ebdff598b0adc9775414901f86871 修复了,你可以更新本项目代码,把http://... 换成 https://... 再试一下。

intfloat avatar Apr 09 '19 03:04 intfloat

你好,换成 https://...后仍然会出现这个问题,__get_request出错.

jyjune24 avatar Apr 09 '19 03:04 jyjune24

  • 项目代码 git pull更新了么?
  • 你把cookie替换成你自己的了么?

intfloat avatar Apr 09 '19 03:04 intfloat

  • 我直接clone到本地环境运行的

  • 是的 已经替换了

jyjune24 avatar Apr 09 '19 03:04 jyjune24

那我想不出来还有什么问题了。。我这边本地测试是好的。

intfloat avatar Apr 09 '19 03:04 intfloat

你好 还想问一下如果账号被微博访问拒绝的话怎么能做到模拟多次尝试连接呢?

jyjune24 avatar May 14 '19 14:05 jyjune24