django-dynamic-scraper
django-dynamic-scraper copied to clipboard
added ProcessBasedUtils to run spiders in-process
I was looking for a way to run the spiders in-processes and avoid scrapyd altogether. I looked online and found a way to do it. If this is something you think is a good idea, I'll be happy to provide more docs.