feat: Poke at found domain variant with web requests

[kristovatlas]

Most domain variants in my experience are boring squatted domains, and could be easily differentiated with a simple HTTP request to http://example.com/.

This could be further extended to support customization by the user to look for the presence or absence of certain tell-tale strings in webpages and for other URL patterns, e.g. to detect commonly known phishing kit installations.

[kristovatlas]

Another reason why this would be a useful feature based on my recent observations, @0xd34db33f: Phishing sites sometimes start out appearing as squatted domains and later change into phishing sites later on without changing DNS settings. It would be cool if this tool could optionally monitor sites over time and alert to changes aside from purely DNS-based ones.

[0xd34db33f]

Love the idea, should be pretty trivial using requests library, md5'ing the downloaded information, and then checking if it's different than the value in a new table we setup. Only tricky part is I've seen a few typo-squatters at $dayjob where they are filled with dynamic content. Wonder if we tweak it so util.py can have an option to disable it if the domain is problematic?

[0xd34db33f]

Started coding this up using Scrapy a little bit ago. Think I've got something, so I'll take ownership for now.