WebCrawler
WebCrawler copied to clipboard
Just a simple web crawler which return crawled links as IObservable using reactive extension and async await.
WebCrawler
Just a simple web crawler which return crawled links as IObservable<Uri> using reactive extension, async await and polly.
dotnet add package MisterHex.WebCrawling --version 2.0.3
Usage
Crawler crawler = new Crawler();
IObservable<Uri> observable = crawler.Crawl(new Uri("https://dotnet.microsoft.com"));
observable.Subscribe(onNext: Console.WriteLine, onCompleted: () => Console.WriteLine("Crawling completed"));