Make Echidna status checking tolerate transient failure
It often happens that checking the Echinda status too quickly after request upload results in failure (to get the validation results), but checking it shortly after that would result in success. So, make the status-check script retry a couple of times on failure.
Weird that the retries are failing with different errors, but clicking the link in a real browser works just fine.
Oh dear, maybe I've killed Echidna again...
@dschuff, what's the status of this PR?
IIRC I never got it quite working reliably. Before this, I switched the echinda upload from completely async (where we didn't check the results) to an attempt to verify that the upload actually succeeded, and IIRC it seemed to work for a while but then stopped working. You may have noticed how often the CI is red. If we want to revert that to what it was before, that would be ok with me. I should probably check that the upload is eventually succeeding though. I suspect we may just have such a large spec that we are causing timeouts in some part of the validation pipeline. I can take a look at the W3C output again and see if I can test the publishing pipeline. This was also all before spectec though. Have you looked at the W3C output from the spectec build at all?
The js-api and web-api failures are real, they should be fixed by https://github.com/speced/bikeshed-boilerplate/pull/173
OK, let's review and land this one. I added a mechanism to ignore persistent failure, and use it for the core spec for now (and with #2000 the web and js-api specs should work correctly). I also filed https://github.com/w3c/echidna/issues/1279 to try to get Echidna fixed.