[v4] `Error: Codecov token not found` in a PR from a fork to a pubilc repo

[jishnub]

My understanding was that a token would not be required when creating a PR from a fork to a public, open-source project. However, in the run https://github.com/JuliaMath/FFTW.jl/actions/runs/7854977161/job/21436255573?pr=295, I encounter

Run codecov/codecov-action@v4
  with:
    fail_ci_if_error: true
    file: lcov.info
  env:
    JULIA_NUM_THREADS: 2
    JULIA_PKG_SERVER_REGISTRY_PREFERENCE: eager
==> windows OS detected
https://cli.codecov.io/latest/windows/codecov.exe.SHA256SUM
==> Running version latest
==> Running version v0.4.6
==> Running command 'D:\a\_actions\codecov\codecov-action\v4\dist\codecov.exe create-commit'
D:\a\_actions\codecov\codecov-action\v4\dist\codecov.exe create-commit -C 798f370e7c6e3cf60fbf6866b3a1b9d36e4db012 -Z
gpg: directory '/c/Users/runneradmin/.gnupg' created
gpg: keybox '/c/Users/runneradmin/.gnupg/pubring.kbx' created
gpg: /c/Users/runneradmin/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1

info - 2024-02-10 12:50:54,352 -- ci service found: github-actions
gpg: Signature made Fri Feb  2 14:15:51 2024 CUT
gpg:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C  62FF 806B B28A ED77 9869

==> Uploader SHASUM verified (64a905e31c4ab88dd999052685656280bc68dafc6bbaaed941a68e6d6a00f24c  codecov.exe)
warning - 2024-02-10 12:50:54,410 -- No config file could be found. Ignoring config.
Error: Codecov token not found. Please provide Codecov token with -t flag.
Error: Codecov: Failed to properly create commit: The process 'D:\a\_actions\codecov\codecov-action\v4\dist\codecov.exe' failed with exit code 1

Other runs in the matrix do succeed without a token, e.g. https://github.com/JuliaMath/FFTW.jl/actions/runs/7854977161/job/21436254781?pr=295, as is expected.

[rohan-at-sentry]

@jishnub thanks for reaching out. We can take a look

[rohan-at-sentry]

@jishnub are you able to request the failing run to be re-run just to confirm?

We suspect that at the time the action ran, we hit Github rate limits which caused our check (to verify if the source was a fork) to fail

[clemenscodes]

Getting the same error on a public repo as well when using v4, using v3 works fine.

[JoseLion]

I'm getting the same issue when using the workflow with workflow_call: from another workflow in the same repo. Public repo as well, and I do have the token: ${{ secrets.CODECOV_TOKEN }} argument set.

I noticed that when the workflow is triggered by push: or pull_request:, the token argument is present:

But when the workflow is triggered by workflow_call: from another workflow, the token argument is missing:

[rohan-at-sentry]

@clemenscodes can you point me to the run where this happened?

[clemenscodes]

@rohan-at-sentry this the run in which it happened: https://github.com/clemenscodes/draken/actions/runs/7890558172/job/21532976187

[jishnub]

@jishnub are you able to request the failing run to be re-run just to confirm?

I can confirm re-running the job does not lead to the issue, so it could certainly be a rate limit. It's just a bit puzzling when jobs fail at random. Perhaps the error message may be improved?

[rohan-at-sentry]

Thanks @jishnub - yeah we're working on fixing the error messaging around rate limits (not just on the CLI, but all other user facing touchpoints - PR, UI etc)

[rohan-at-sentry]

@JoseLion @clemenscodes if you can open a different issue, while I take a look that would be appreciated. This will help me track better. FWIW @clemenscodes - my understanding is that it looks to be separate from this issue (which is about forks).

Closing this as the original issue is resolved

[georgebv]

Getting the same error in my own public repo when running v4 triggered by a release event on main branch. You can see verbose logs here: https://github.com/georgebv/drf-pydantic/actions/runs/7896620245/job/21550933382

As you can see, re-running (once) didn't help.

Rolling back to v3 solved this.

[terabytesoftw]

The error persists, we had to downgrade to v3.

[wsanchez]

Getting hit by this also: https://github.com/burningmantech/ranger-ims-server/actions/runs/7866907121/job/21574339078?pr=1116

[kiraware]

Me to here https://github.com/kiraware/paspybin/actions/runs/7911211535/job/21595033660. It would be nice if codecov failing the workflow not just passed as everything is ok.

[carlosmondra]

Me to here https://github.com/kiraware/paspybin/actions/runs/7911211535/job/21595033660. It would be nice if codecov failing the workflow not just passed as everything is ok.

@kiraware Are you passing the param fail_ci_if_error: true?

[kiraware]

Me to here https://github.com/kiraware/paspybin/actions/runs/7911211535/job/21595033660. It would be nice if codecov failing the workflow not just passed as everything is ok.

@kiraware Are you passing the param fail_ci_if_error: true?

@carlosmondra nice point, i miss the docs. Thanks for pointing it.

And for everyone, I was surprised because codecov gave the wrong guide, here is an example of the guide I got when setting up a new repo

as you can see, it should using with instead of env. Based on the readme stated

steps:
- uses: actions/checkout@master
- uses: codecov/codecov-action@v4
  with:
    fail_ci_if_error: true # optional (default = false)
    files: ./coverage1.xml,./coverage2.xml # optional
    flags: unittests # optional
    name: codecov-umbrella # optional
    token: ${{ secrets.CODECOV_TOKEN }} # required
    verbose: true # optional (default = false)

and via environment variable

steps:
- uses: actions/checkout@master
- uses: codecov/codecov-action@v4
  with:
    fail_ci_if_error: true # optional (default = false)
    files: ./coverage1.xml,./coverage2.xml # optional
    flags: unittests # optional
    name: codecov-umbrella # optional
    verbose: true # optional (default = false)
  env:
    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

Again, it was my fault not read the docs. But the guide needs to be fixed soon. Thanks.

[wheeheee]

On the DSP.jl repo, this happens suspiciously often on macos-latest. 5 / 6 of the most recent Codecov failures are on this platform (2 in a matrix of 8). https://github.com/JuliaDSP/DSP.jl/actions/runs/7989226135/job/21815388948 (some are hidden behind re-runs)

[jayqi]

I'm getting the Error: Codecov token not found. on a PR from Dependabot on my repo. I'm not exactly sure how Dependabot works, but GitHub Actions appears to be treating it as a public fork as it's not passing in my token. I've tried rerunning my workflow five times over 3 days, and I'm still getting an error.

https://github.com/jayqi/failed-build-issue-action/actions/runs/7964800022/job/21822931777

https://github.com/jayqi/failed-build-issue-action/blob/92418847f91ab409c250460723b5b5f5487a8d48/.github/workflows/tests.yml#L31-L35

[kiraware]

@jayqi Hello, I have experienced this too. That's because dependabot is not allowed to read Github Action Secrets based on this documentation. If you put CODECOV_TOKEN in the Github Secret then dependabot definitely won't be able to read the secret. So that CODECOV_TOKEN can be read by dependabot, you need to put it in Dependabot Secret.

Navigate to the repository settings, then look in the left panel in the "Secrets and variables" section, there is a dropdown option, now you select Dependabot and add CODECOV_TOKEN there. Hope that helps solve your problem.

[ldemailly]

is there a reason for why we need a token for public repos while we didn't before? that's creating a lot of churn/changes

[kristof-mattei]

@JoseLion to get secrets passed on to a workflow_call you need to expose the secrets explicitly, either individually or as inherit: https://docs.github.com/en/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

I just ran into the same issue.

[Czaki]

is there a reason for why we need a token for public repos while we didn't before? that's creating a lot of churn/changes

They hit in limits of calling GitHub API required for identifying workflow run. If you upload with a token, they save GitHub API calls. Best will be if they will have higher limit, but it depends on GitHub itself.

[superical]

After spending over an hour grappling with this issue, it seems like my problem was with the token (which is stored as a secret) missing in codecov when triggering via a workflow_call. I'm not sure why it has worked all along until today.

For anyone facing the same problem, my workaround is passing the secret from the caller job into the workflow job. This allows codecov to receive the token. Here's the change that did the trick for me: https://github.com/Open-Attestation/document-store/pull/170/files

I believe you only have to pass the token input to codecov if you aren't using a workflow_call like I did.

[Czaki]

For anyone facing the same problem, my workaround is passing the secret from the caller job into the workflow job

It is not workaround but official way to do this (I face same problem a week ago) https://github.com/napari/napari/pull/6701/files

It is in documentation, but not clearly written.

[superical]

Ah... Glad to know that it's the official way to do it.👍

Is this even mentioned in the documentation? I didn't find anything about this issue. I ended up trying different methods just to pass the token into the action.

[Czaki]

https://docs.github.com/en/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onworkflow_callsecrets

Note: If you are passing the secret to a nested reusable workflow, then you must use jobs.<job_id>.secrets again to pass the secret. For more information, see "Reusing workflows."

[thomasrockhu-codecov]

I think the original issue here has been resolved so closing for now. If there is still more to respond to, please create a new issue.