locust icon indicating copy to clipboard operation
locust copied to clipboard

Published 20 hours ago • verified publisher icon achannarasappa
→

Metadata

Distributed web data discovery and collection framework built for serverless

Build Status Coverage Status

Locust

Distributed web data discovery and collection framework

Quick Start

npm install @achannarasappa/locust

Features

  • Configuration driven jobs
  • Distributed execution model to support serverless architectures
  • Handle client-side JavaScript execution
  • Data extraction using CSS selectors
  • Depth-based stop condition along with support for custom stop conditions
  • Robust dev tooling with locust-cli to build and test jobs

Use Cases

  • Web indexing (i.e. web crawling)
  • Web data extraction (i.e. web scraping)

Reference

← Metadata

16
Stars
1
Forks
Watchers

Owner

verified publisher icon achannarasappa

Metadata

Distributed web data discovery and collection framework built for serverless

Back