mkdocs-htmlproofer-plugin icon indicating copy to clipboard operation
mkdocs-htmlproofer-plugin copied to clipboard

Published 20 hours ago • verified publisher icon manuzhang

Question: how to handle/ignore `-1` errors

Open emmercm opened this issue 1 year ago • 8 comments

Describe the bug

I'm getting a -1 error that I'm assuming means a timeout:

 WARNING  -  htmlproofer: invalid url - http://www.logiqx.com/DatFAQs/CMPro.php [-1] [dats.md]

I can't seem to ignore this in my mkdocs.yml:

plugins:
  - htmlproofer:
      raise_error_excludes:
        '-1': [ 'http://www.logiqx.com' ]

To Reproduce

Add a link to http://www.logiqx.com/DatFAQs/CMPro.php or some other website that's timing out and then run mkdocs.

Environment (please complete the following information):

  • OS (e.g. Windows 10/Ubuntu 20.04/macOS): macOS Ventura 13.2.1
  • Python version (python --version): v3.10.8
  • Package versions (pip list): v0.12.0

Contents of mkdocs.yml:

plugins:
  - htmlproofer:
      raise_error_excludes:
        '-1': [ 'http://www.logiqx.com' ]

emmercm avatar Apr 16 '23 21:04 emmercm

do you have another example? This site doesn't timeout for me.

manuzhang avatar Apr 18 '23 15:04 manuzhang

They must have fixed whatever was wrong.

There are websites that can be used to simulate the timeout, e.g. https://httpstat.us/200?sleep=60000 with a 60sec response delay.

emmercm avatar Apr 18 '23 15:04 emmercm

I still cannot reproduce the issue. Can you make a PR adding your test case to our integration tests?

manuzhang avatar Apr 19 '23 01:04 manuzhang

I spent some time looking and I couldn't find any website that could simulate TooManyRedirects or ConnectionError. The best I found was https://httpstat.us/ which is just status codes.

I think this will need to be simulated in unit tests, where those exceptions are raised by the mocks. I'm not super familiar with unit testing in Python, I would need help there.

emmercm avatar Apr 20 '23 14:04 emmercm

Can you firstly add your case to https://github.com/manuzhang/mkdocs-htmlproofer-plugin/tree/main/tests/integration?

manuzhang avatar Apr 21 '23 14:04 manuzhang

Adding http://www.logiqx.com wouldn't be a good test, as it looks like the website has since recovered and is not producing the error anymore. I think the same would be true of most real websites, I would expect the -1 code to be transient.

I could not find a website used for HTTP testing (e.g. https://httpstat.us/ mentioned above) that could produce a -1 code reliably, and I went searching for a while.

emmercm avatar Apr 21 '23 14:04 emmercm

@emmercm Did you ever get to the bottom of this issue? I ask because we are running into the exact same thing, with the exact same circumstances:

  1. www.logiqx.com is a DNS CNAME record; it points to logiqx.com.
  2. logiqx.com. has A records, but no AAAA records.

I see the same intermittent failures that you saw, against a host for which both these criteria are also true.

Now this kind of scenario shouldn't be a problem. But I am beginning to surmise that what we're dealing with may be an issue with the requests module, or beneath it the DNS resolution stack, where some moving part is somehow getting confused when running on a dual-stack (IPv4/IPv6) client. Would you reckon that that might be a possibility?

fghaas avatar Jun 29 '23 18:06 fghaas

We are getting these fairly regularly, the -1 but it isn't reproducible for any individual URL.

If anyone has figured out a way to ignore the -1 entries, I'd love to hear about it.

If there is anything I could provide to help with this, please let me know.

SeanTAllen avatar Feb 13 '24 19:02 SeanTAllen