htmlSanityCheck icon indicating copy to clipboard operation
htmlSanityCheck copied to clipboard

Published 20 hours ago • verified publisher icon aim42

avoid bad 403 and 405 results

Open gernotstarke opened this issue 7 years ago • 4 comments

e.g. Amazon always returns 405 upon HEAD requests.

We should send a GET after all suspicious error codes (esp. 403 and 405) to get better results.

gernotstarke avatar Oct 04 '18 09:10 gernotstarke

should be fixed now on branch 1.0.0-RC-2

gernotstarke avatar Oct 10 '18 07:10 gernotstarke

the test currently breaks. Could it be that amazon has some kind of crawler protection? The checker currently seems to get a 503 when the tests are run through travis or locally... Maybe we need to send a fake user agent in addition to get through...

rdmueller avatar Oct 14 '18 19:10 rdmueller

current workaround: ignore automated test and ignore false negatives in test result

rdmueller avatar Oct 14 '18 19:10 rdmueller

added some special-case handling (add user-agent to connection), reducing the number of wrong 403's for some servers.

gernotstarke avatar Oct 26 '18 14:10 gernotstarke