web-crawlers topic

List web-crawlers repositories

isbot

848
Stars
73
Forks
Watchers

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string

scrape-amazon

32
Stars
11
Forks
Watchers

🤩 Python Package for Scraping Amazon Product Reviews ✨

amazon_scraper

76
Stars
20
Forks
Watchers

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

url-frontier

40
Stars
9
Forks
Watchers

API definition, resources and reference implementation of URL Frontiers

learnpython2018

23
Stars
25
Forks
Watchers

Python course for 2nd year NLP students at NRU HSE, 2018-2019

iot-tree

69
Stars
24
Forks
Watchers

A IOT-Server,The tree root supports devices and data management, and the upper tree supports to combine various business functions by message flow

gamechanger-crawlers

15
Stars
9
Forks
Watchers

GAMECHANGER Policy Analytics Site Crawlers

learnpython2017

15
Stars
7
Forks
Watchers

Python course for 2nd year NLP students at NRU HSE, 2017-2018