webcrawler icon indicating copy to clipboard operation
webcrawler copied to clipboard

Feature: implement a retry to go over the links that failed or errors

Open verovaleros opened this issue 1 year ago • 0 comments

If the connection is lost, the HTTP requests will fail. There are a few ways to handle this:

  • Identify the issue and re-add the link to the queue
  • Add a retry option to go over the links that failed or threw an error

verovaleros avatar Dec 30 '23 10:12 verovaleros