RISJbot icon indicating copy to clipboard operation
RISJbot copied to clipboard
verified publisher icon pmyteh

Spider not found

Open smyja opened this issue 4 years ago • 1 comments

I keep getting key error spider not found:CNN when I run scrapy crawl cnn or for any news website. What directory am I supposed to run that in? The README is very vague.

smyja avatar Sep 21 '21 13:09 smyja

If you're getting 'Spider not found' then scrapy itself is running. Have you installed scrapy in a venv, as recommended by their installation docs? Can you run any of the other spiders? The 'base' directory to look for spiders is found in your settings.py file, which should draw from RISJbot.spiders by default. And everything is case sensitive.

pmyteh avatar Sep 22 '21 17:09 pmyteh