scrapy-do
scrapy-do copied to clipboard
payload is literally provided as payload in the kwargs of spider
Wondering if it's a bug or not that payload
is literally provided to the kwargs as payload
. Expected is a dict of arguments as Scrapy documented: https://docs.scrapy.org/en/latest/topics/spiders.html#spider-arguments
Payload:
{
"start_url": "https://books.toscrape.com/"
}
Spider:
def __init__(self, start_url=None, *args, **kwargs):
print("args")
print(args)
print("kwargs")
print(kwargs)
if not start_url:
raise ValueError("argument string 'start_url' must be provided!")
self.start_urls = [start_url]
Output:
args
()
kwargs
{'payload': '{"start_url": "https://books.toscrape.com/"}'}