arachne
arachne copied to clipboard
Item_pipeline not working
I have made a news scraping spider which stores items into an sqlite3 database. I've added the following to the settings.py file
SPIDER_SETTINGS = [ { 'endpoint': 'tech_news', 'location': 'spiders.news_spider', 'spider': 'TechSpider', 'scrapy_settings': { 'ITEM_PIPELINES': { 'pipelines.NewsCrawlerPipeline': 300 } } } ]
But the Item_pipeline doesn't seem to be working as there is no data being sent to the database. The spider is working perfectly fine when I run it through the terminal.