scrapy-distributed
scrapy-distributed copied to clipboard
Published
20 hours ago •
Insutanto
Insutanto
A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
Results
10
scrapy-distributed issues
Sort by
recently updated
recently updated
newest added
Can you add dynamic web crawlers into your project? Need to use simulation click technology and anti-climbing mechanism. Thx！
feature
Hi @Insutanto Great! Your work is really impressive. However, I would like to add some suggestions. First of all, I wanna open the front console of RabbitMQ(http://127.0.0.1:15672), but it didn't...
enhancement
good first issue
feature
Hi @Insutanto You doing nice work in this repo. I have the same desire: different message queues should be supported in scrapy. Old implementations of this idea and one you...
enhancement