[FEATURE REQ] Rate limit requests to the same host

[hugobuddel]

Is your feature request related to a problem? Please describe.

I've got a README.md that has two links to a hackernews post. Since a couple of days, linkspector fails with a 503 on the 2nd URL in the file, independently of the order. Linkspector succeeds when either of the URL's is commented out.

It therefore seems that the server is responding witht 503 because linkspector fires the request to quickly, or something like that.

Describe the solution you'd like

Perhaps linkspector could wait a (configurable) amount of time between requests to the same host.