tiny-web-crawler icon indicating copy to clipboard operation
tiny-web-crawler copied to clipboard

verified publisher icon indrajithi
→

Metadata

A tiny web crawler in Python

A Tiny Web Crawler

A web crawler written in python.

Install Requirements

pip install validators beautifulsoup4 lxml

Python version: Python 3.6.3 :: Anaconda, Inc.

Run

python crawler.py

Starts crawling from root_url until max_link urls are crawled. Saves the output to out.json.

← Metadata

36
Stars
8
Forks
Watchers

Owner

verified publisher icon indrajithi

Metadata

A tiny web crawler in Python

Back