crawling topic

Declarative web scraping

MontFerret

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o...

apify

A Devtools driver for web automation and scraping

go-rod

네이버 뉴스 수집을 위한 도구

forkonlp

ISP Data Pollution to Protect Private Browsing History with Obfuscation

essandess

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

codelucas

HTTP API for Scrapy spiders

scrapinghub

Scrapy middleware to handle javascript pages using selenium

clemfromspace

Second-order subdomain takeover scanner

mhmdiaa

Simple but useful Python web scraping tutorial code.

MorvanZhou