CoCrawler

icon location CoCrawler is a modern web crawling framework written in Python's new coroutine syntax.

Results 2 repositories owned by CoCrawler

cocrawler

177
Stars
25
Forks
Watchers

CoCrawler is a versatile web crawler built using modern tools and concurrency.

verified publisher icon cocrawler

cdx_toolkit

153
Stars
29
Forks
Watchers

A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine

verified publisher icon cocrawler