Crawl a Site that requires Credentials

[LiveBacteria]

Can a feature/flag be added to allow for crawling sites that need credentials for accessing specific pages?

[steve8708]

Since this project uses a headless browser that should be achievable and would make for a great PR if you are interested in contributing it!