PHP-CSS-Parser icon indicating copy to clipboard operation
PHP-CSS-Parser copied to clipboard

Code coverage server is unreliable

Open JakeQZ opened this issue 10 months ago • 5 comments

CI and PR checks should not fail if Coveralls is offline. This has happened many times recently. It's unworkable as it is.

We need to find a way of disregarding any error from Coveralls, and just treating it as missing information instead.

JakeQZ avatar Apr 07 '25 01:04 JakeQZ

https://github.com/actions/runner/issues/2347 has some relevant but rather too much discussion. There might a needle in a haystack in there.

JakeQZ avatar Apr 07 '25 01:04 JakeQZ

allow-failure flag may be possible - https://github.com/actions/runner/issues/2347#issuecomment-1367587244

Edit: Not it isn't. That was just another rejected suggestion.

JakeQZ avatar Apr 07 '25 03:04 JakeQZ

At the moment, the code coverage is not as required in the branch protection settings, i.e., it should be possible to merge a PR even if the code coverage has that angry red dot (and despite the failure email). Does this work for you, @JakeQZ ?

oliverklee avatar Apr 07 '25 17:04 oliverklee

branch protection settings

Any PR can be merged with failing tests by clicking the "Override branch protection settings" link/button.

IIRC, the Coveralls failure meant the whole CI was 'red', and would need such override to merge. But my memory is not what it used to be, and these were PRs I submitted rather than reviewed. If you were able to merge despite the Coveralls failure, and without overriding branch protection, then I guess this is passable (though not ideal).

If OTOH you had to override branch protection, there may be dragons...

https://github.com/actions/runner/issues/2347 was closed as 'Not Planned', and I did not find a solution in the comments. So there may be no other way for now.

PS. Coveralls seems to be back up and running fine and dandy now, so the situation is no longer (easily) reproducible.

JakeQZ avatar Apr 08 '25 00:04 JakeQZ

Any PR can be merged with failing tests by clicking the "Override branch protection settings" link/button.

Yes, that's true.

If the code coverage job fails, it's possible to use the regular merge button, though (as the "override" button is only necessary if a required job fails).

oliverklee avatar Apr 08 '25 08:04 oliverklee