cloudflare-scrape icon indicating copy to clipboard operation
cloudflare-scrape copied to clipboard

Why doesn't cfscrape work in scrapy

Open glfxml opened this issue 5 years ago • 0 comments
trafficstars

My code is as follows,but it returns 403

def start_requests(self):
    user_agent = self.settings['USER_AGENT']

    for url in self.start_urls:
        tokens, agent = cfscrape.get_tokens(url, user_agent)
        yield scrapy.Request(url=url,
                                   cookies={'__cfduid': tokens['__cfduid']},
                                   headers={'User-Agent': agent})

glfxml avatar Aug 10 '20 09:08 glfxml