scrapy
scrapy copied to clipboard
Process cookies from header only once
:construction: Work in progress :construction:
A first approach to solving #4717. The idea is to process cookies from the header only once, the actual method of solving it can vary (meta key, request attribute, etc). I know it's not much, but I'd like your opinions @Gallaecio @kmike @wRAR.
The following spider works with this patch, while it breaks with the current master (5a386393) - I'm working on some tests to reproduce this behaviour.
import scrapy
class LoginSpider(scrapy.Spider):
name = "login"
start_urls = ["http://quotes.toscrape.com/login"]
custom_settings = {"COOKIES_DEBUG": True}
def parse(self, response):
csrf_token = response.xpath("//input[@name='csrf_token']/@value").get()
return scrapy.FormRequest.from_response(
response=response,
formxpath="//form",
formdata={
"csrf_token": csrf_token,
"username": "admin",
"password": "admin"
},
callback=self.after_login,
)
def after_login(self, response):
logout = response.xpath("//a[text()='Logout']/text()").get()
assert logout is not None
Fixes #1992