cobweb
cobweb copied to clipboard
Redirect Limit causing crawl to stop
When the redirect limit is hit it kills the crawl. The RedirectError is thrown but doesn't seem to be trapped, it seems to be thrown for each subsequent call into the get method, which it shouldn't do because it should check that the redirect_limit has count down to 0.