scraping topic

List scraping repositories

crawler-chrome-extensions

68
Stars
10
Forks
Watchers

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

html-table-to-json

40
Stars
7
Forks
Watchers

Generate JSON representations of HTML tables

mtnt

55
Stars
4
Forks
Watchers

Code for the collection and analysis of the MTNT dataset

pdf.tocgen

571
Stars
24
Forks
Watchers

A CLI toolset to generate table of contents for PDF files automatically.

worldometer

55
Stars
9
Forks
Watchers

Get live, population, geography, projected, and historical data from around the world 🌍

requests-html

13.6k
Stars
978
Forks
Watchers

Pythonic HTML Parsing for Humans™

freesoccer

36
Stars
9
Forks
Watchers

:soccer: Free API with results from national soccer competitions

Humanoid

202
Stars
27
Forks
Watchers

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

RARBG-scraper

42
Stars
11
Forks
Watchers

With Selenium headless browsing and CAPTCHA solving

List-of-user-agents

419
Stars
223
Forks
Watchers

List of major web + mobile browser user agent strings. +1 Bonus script to scrape :)