scrapy-do icon indicating copy to clipboard operation
scrapy-do copied to clipboard

payload is literally provided as payload in the kwargs of spider

Open dot-mike opened this issue 4 years ago • 0 comments

Wondering if it's a bug or not that payload is literally provided to the kwargs as payload. Expected is a dict of arguments as Scrapy documented: https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments

Payload:

{
    "start_url": "https://books.toscrape.com/"
}

Spider:

    def __init__(self, start_url=None, *args, **kwargs):
        print("args")
        print(args)
        print("kwargs")
        print(kwargs)
        if not start_url:
            raise ValueError("argument string 'start_url' must be provided!")
        self.start_urls = [start_url]

Output:

args
()
kwargs
{'payload': '{"start_url": "https://books.toscrape.com/"}'}

dot-mike avatar Jan 28 '21 02:01 dot-mike