BlackWidow

Web crawler for HeelsFetishism

Install

$ sudo apt-get install python-dev libxml2-dev libxslt1-dev
$ pip install -r requirements.txt

Usage

$ scrapy crawl atlanticpacific
$ scrapy crawl beautylegmm
$ scrapy crawl fancy
$ scrapy crawl garypeppergirl
$ scrapy crawl itscamilleco
$ scrapy crawl madamejulietta
$ scrapy crawl ohmyvogue
$ scrapy crawl pinterest
$ scrapy crawl sayhellomax
$ scrapy crawl tvcclub
$ scrapy crawl wendyslookbook