2 icon indicating copy to clipboard operation
2 copied to clipboard

网页不存在了代码怎么修改

Open kevinGGL opened this issue 6 years ago • 2 comments

来自一个菜鸟的提问,http://www.mmjpg.com/home/这个网页不存在了,但是http://www.mmjpg.com/还是可以看,那代码应该做怎样的修改?

kevinGGL avatar May 24 '18 07:05 kevinGGL

网站采取了一点反爬虫措施,必须要加上“请求体”(概念请百度自行搜索):

headers={ 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0', 'Referer':'http://www.mmjpg.com/' } 然后把 response = requests.get(url).content 改为 response = requests.get(url,headers=headers,timeout=10).content User-Agent:告诉网站我是谁; Referer:告诉网站我从哪里来。 对不起,我们骗了网站!!!

fmscole avatar May 25 '18 14:05 fmscole

增加headers了之后,爬下来的图片都是防盗图了楼主。该如何反爬呢?

lxnba66 avatar Jun 15 '18 08:06 lxnba66