scrapy icon indicating copy to clipboard operation
scrapy copied to clipboard

Process cookies from header only once

Open elacuesta opened this issue 3 years ago • 7 comments

:construction: Work in progress :construction:

A first approach to solving #4717. The idea is to process cookies from the header only once, the actual method of solving it can vary (meta key, request attribute, etc). I know it's not much, but I'd like your opinions @Gallaecio @kmike @wRAR.

The following spider works with this patch, while it breaks with the current master (5a386393) - I'm working on some tests to reproduce this behaviour.

import scrapy

class LoginSpider(scrapy.Spider):
    name = "login"
    start_urls = ["http://quotes.toscrape.com/login"]
    custom_settings = {"COOKIES_DEBUG": True}

    def parse(self, response):
        csrf_token = response.xpath("//input[@name='csrf_token']/@value").get()
        return scrapy.FormRequest.from_response(
            response=response,
            formxpath="//form",
            formdata={
                "csrf_token": csrf_token,
                "username": "admin",
                "password": "admin"
            },
            callback=self.after_login,
        )

    def after_login(self, response):
        logout = response.xpath("//a[text()='Logout']/text()").get()
        assert logout is not None

Fixes #1992

elacuesta avatar Sep 29 '20 17:09 elacuesta