Send pending status checks on initial receipt of coverage reports

[heaths]

I have codecov/project and codecov/patch set up as required checks and my repo configured to require certain thresholds. The problem is that those aforementioned checks don't always start soon enough and an otherwise successful PR - with reduced code coverage - may be merged using GitHub's auto-merged.

What would be ideal is that when an upload is received via the codecov/codecov-action@v3, it immediate posts back to the PR that a check is running even if it hasn't begun processing. This should be fast enough to allow required branch checks to work as desired.

Example workflow: https://github.com/heaths/azcrypto/blob/5fb26e6cc58cd9743ecf0c2cdf9c9db268a25bea/.github/workflows/ci.yml#L49

[thomasrockhu-codecov]

@heaths going to move this over to our feedback repository.

@rohan-at-sentry this could be an interesting idea, we'd have to ensure that after_n_builds is set though

[drazisil-codecov]

@heaths ,

Required checks is a GitHub feature, and GitHub will create that pending check for an check set as required. I looked at a recent PR

And it looks like you only have one of your CI jobs set as required. Can you double-check your settings?

[heaths]

Interesting. It was at least back then. Might've changed the config at some point. Still, GitHub won't auto-merge if checks are still running. So whether or not a repo would require these checks at a certain threshold, I don't think anyone would want their PRs auto-merging if any check - required or not - is still running. I will ensure that these are required checks, though.

[rohan-at-sentry]

@heaths did setting the checks are required help at all?

[heaths]

I remembered that, at the time, it was a team decision not to require a certain code coverage threshold at this time. We just want to make sure they are running and reporting, and tend to clean it up in separate quality milestones if time is tight. @jsquire do you want to revisit whether code coverage should be a required check and adjust thresholds accordingly?

[rohan-at-sentry]

what you could do is set Codecov as a required check on Github (that way you need that check to run), and then configure Codecov to run on informational mode if you're not quite ready to block