web-scraping topic
Humanoid
Node.js package to bypass CloudFlare's anti-bot JavaScript challenges
halfstaff
:us: Is the US flag at half-staff?
DAT8
General Assembly's 2015 Data Science course in Washington, DC
fb_friend_list_scraper
OSINT tool to scrape names and usernames from large friend lists on Facebook, without being rate limited.
arachnid
Powerful web scraping framework for Crystal
snoop
Snoop — инструмент разведки на основе открытых данных (OSINT world)
scrapy-wayback-machine
A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
htmldate
Fast and robust date extraction from web pages, with Python or on the command-line
trafilatura
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dude
dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators