Codecov not reporting back to Github Pull Request checks

[conbrad]

Describe the bug Codecov not reporting back to Github Pull Request checks, even though report is successfully uploaded. Checks always remain in "Expected" state.

Environment (please complete the following information):

• System: Codecov Github Action, codecov/codecov-action@v4

To Reproduce Steps to reproduce the behavior:

1. Action runs fine, e.g. https://github.com/bcgov/wps/actions/runs/8086129987/job/22095603938?pr=3422
2. Results are on Codecov, e.g. https://app.codecov.io/github/bcgov/wps/commit/354c7a21c6c34060e2000dd9d486ee72cd037232
3. Checks never move from expected to success state, e.g.

Expected behavior A successful Codecov processed run should update the checks

Additional context

• Can view results on codecov page
• Validated yaml here with:

curl -X POST --data-binary @projects/wps/codecov.yml https://codecov.io/validate
Valid!

{
  "ignore": [
    "^web/src/features/cHaines.*",
    "^web/src/features/fwiCalculator.*",
    "^web/src/features/fba.*",
    "^web/src/api/fbaAPI.ts.*",
    "^web/src/utils/env.ts.*",
    "^api/app/auto_spatial_advisory.*",
    "^api/app/db/crud.*",
    "^api/app/db/database.py.*",
    "^api/app/tests.*"
  ],
  "coverage": {
    "status": {
      "patch": {
        "default": {
          "threshold": 100.0
        }
      },
      "project": {
        "default": {
          "target": 80.0,
          "threshold": 3.0
        },
        "api": {
          "paths": [
            "^api/app.*"
          ],
          "threshold": 3.0
        },
        "web": {
          "paths": [
            "^web/src.*"
          ],
          "target": 73.0
        }
      }
    }
  }
}

[bnjbvr]

Same here, this blocks most of our pull requests, in case you need more data from another public repository here's ours: https://github.com/matrix-org/matrix-rust-sdk

[rohan-at-sentry]

@drazisil-codecov can you help out here?

[drazisil-codecov]

@conbrad Do you have the Codecov GitHub app installed? https://github.com/apps/codecov

Looks like there was a permissions error where Codecov couldn't send the checks. (PR comments can be made by anyone)

@bnjbvr Could you gave me an example of what you are seeing? Do you have the app installed as well?

[conbrad]

@conbrad Do you have the Codecov GitHub app installed? https://github.com/apps/codecov

Looks like there was a permissions error where Codecov couldn't send the checks. (PR comments can be made by anyone)

Hi @drazisil-codecov , yeah we have the Codecov Github app installed. Not sure what permissions changes you mean exactly but here's the current permissions that used to work until recently:

[drazisil-codecov]

Hi @conbrad ,

Is it granted access to the repo in question? Also, we made a change yesterday that might have been related to your issue. Can you try again?

[conbrad]

Hi @conbrad ,

Is it granted access to the repo in question? Also, we made a change yesterday that might have been related to your issue. Can you try again?

Hi @drazisil-codecov

Yes the Codecov app has been granted access and permissions to our repo:

I opened this PR ~20 minutes ago and it's waiting for the expected checks: https://github.com/bcgov/wps/pull/3510

[drazisil-codecov]

@conbrad ,

Got it. Let me dig in with my engineers to see why it's behaving this way.

[eliatcodecov]

This may be a rate limit issue due to using codecov's default commenter instead of the installed app integration. We're currently still investigating what causes this behavior, and will update when we know more.

[gtbuchanan]

This has been a problem for a while. A large number of our pull requests don't get status updates. Will you please provide an update?

Here is an example from one of our private repos:

[conbrad]

FYI this has actually started to work for us

[gtbuchanan]

I discovered that it might only be occurring for our TypeScript repositories that are using codecovcli upload-process directly. Our .NET repositories use CodecovUploader (based on codecov/uploader). I'm not sure if this is truly the cause, but it's the only difference I'm seeing at the moment.

[conbrad]

I discovered that it might only be occurring for our TypeScript repositories that are using codecovcli upload-process directly. Our .NET repositories use CodecovUploader (based on codecov/uploader). I'm not sure if this is truly the cause, but it's the only difference I'm seeing at the moment.

We use the github action, codecov/codecov-action@v4 for our uploads. We also have the Github app installed and perhaps there's some redundancy there but for sure our github action runs.

[drazisil-codecov]

I'm getting confused since there are 3 customers in this thread.

Can I please get a roll call of who is having issues, and a recent SHA (head of PR) if you are still in needs of help? 🙏 ❤️

[gtbuchanan]

@drazisil-codecov I ended up going through Pro support and submitted a pull request here to fix my problem.

EDIT: Whoops, just realized you are the one who handled my request 🙃 Oh well, now people with a similar problem can find it,

[trent-codecov]

It looks like the issues presented in this ticket are resolved. Please open a new ticket referencing this one if there are remaining problems.