codecov-action icon indicating copy to clipboard operation
codecov-action copied to clipboard

[BUG] Codecov failed to associate the commit in a PR with the PR itself.

Open vivodi opened this issue 7 months ago • 4 comments

Describe the bug The coverage for the commit in a PR was successfully uploaded to Codecov via GitHub Actions, and the commit along with its coverage is visible in the Codecov UI.

However, the related PR page in the Codecov UI shows "Missing Head Commit", and the commit section for the PR is empty.

The patch and project coverage checks from Codecov are visible on the GitHub PR page, but there is no comment.

To Reproduce

  1. The Codecov-generated coverage checks are displayed on the GitHub PR page, but no comment was created on the PR: https://github.com/Flexget/Flexget/pull/4367
  2. The coverage information for the commit in the PR is shown in the Codecov UI (https://app.codecov.io/gh/Flexget/Flexget/commit/4ce231c77f758cb4af29b90357b4a52b9c1e9c22), but the PR page shows "Missing head commit", meaning the corresponding commit is not associated with the PR (https://app.codecov.io/gh/Flexget/Flexget/pull/4367).

Expected behavior The corresponding commit can be seen in the PR UI on Codecov, and a comment can be created in the PR.

Regression This is not a regression. Screenshots

The commit page on Codecov correctly displays the commit from the PR: Image

but the PR page fails to show the commit: Image

Product Area coverage

Versions

  • OS: [ubuntu@v24, macos arm64, windows]
  • Git Host: [GitHub]
  • CI/CD: [GitHub Actions]
  • Uploader: [[email protected]]

Commit and CI link commit SHA: 4ce231c77f758cb4af29b90357b4a52b9c1e9c22 CI link: https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366

the entire Codecov output from CI:

Run codecov/codecov-action@ad3126e916f78f00edff4ed0317cf185271ccc2d
  with:
    fail_ci_if_error: true
    name: ubuntu-latest-3.13
    disable_file_fixes: false
    disable_search: false
    disable_safe_directory: false
    disable_telem: false
    dry_run: false
    git_service: github
    gcov_executable: gcov
    handle_no_reports_found: false
    recurse_submodules: false
    run_command: upload-coverage
    skip_validation: false
    use_legacy_upload_endpoint: false
    use_oidc: false
    use_pypi: false
    verbose: false
    version: latest
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
Run CC_ACTION_VERSION=$(cat ${GITHUB_ACTION_PATH}/src/version)
  CC_ACTION_VERSION=$(cat ${GITHUB_ACTION_PATH}/src/version)
  echo -e "\033[0;3[2](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:2)m==>\0[3](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:3)3[0m Running Action version $CC_ACTION_VERSION"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
==> Running Action version 5.[4](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:4).2
Run git config --global --add safe.directory "/home/runner/work/Flexget/Flexget"
  git config --global --add safe.directory "/home/runner/work/Flexget/Flexget"
  git config --global --add safe.directory "$GITHUB_WORKSPACE"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
Run CC_FORK="false"
  CC_FORK="false"
  if [ -n "$GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FULL_NAME" ] && [ "$GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FULL_NAME" != "$GITHUB_REPOSITORY" ];
  then
    echo -e "\033[0;32m==>\033[0m Fork detected"
    CC_FORK="true"
  fi
  echo "CC_FORK=$CC_FORK" >> "$GITHUB_ENV"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
    GITHUB_EVENT_PULL_REQUEST_HEAD_LABEL: vivodi:fix_flush
    GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FULL_NAME: vivodi/Flexget
    GITHUB_REPOSITORY: Flexget/Flexget
==> Fork detected
Run if [ "false" == 'true' ] && [ "$CC_FORK" != 'true' ];
  if [ "false" == 'true' ] && [ "$CC_FORK" != 'true' ];
  then
    echo "CC_TOKEN=$CC_OIDC_TOKEN" >> "$GITHUB_ENV"
  elif [ -n "" ];
  then
    echo -e "\033[0;32m==>\033[0m Token set from env"
      echo "CC_TOKEN=" >> "$GITHUB_ENV"
  else
    if [ -n "" ];
    then
      echo -e "\033[0;32m==>\033[0m Token set from input"
      CC_TOKEN=$(echo "" | tr -d '\n')
      echo "CC_TOKEN=$CC_TOKEN" >> "$GITHUB_ENV"
    fi
  fi
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
    CC_FORK: true
    CC_OIDC_TOKEN: 
    CC_OIDC_AUDIENCE: https://codecov.io
Run if [ -z "$CC_BRANCH" ] && [ -z "$CC_TOKEN" ] && [ "$CC_FORK" == 'true' ]
  if [ -z "$CC_BRANCH" ] && [ -z "$CC_TOKEN" ] && [ "$CC_FORK" == 'true' ]
  then
    echo -e "\033[0;32m==>\033[0m Fork detected, setting branch to $GITHUB_EVENT_PULL_REQUEST_HEAD_LABEL"
    TOKENLESS="$GITHUB_EVENT_PULL_REQUEST_HEAD_LABEL"
    CC_BRANCH="$GITHUB_EVENT_PULL_REQUEST_HEAD_LABEL"
    echo "TOKENLESS=$TOKENLESS" >> "$GITHUB_ENV"
  fi
  
  echo "CC_BRANCH=$CC_BRANCH" >> "$GITHUB_ENV"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
    CC_FORK: true
    CC_BRANCH: 
    GITHUB_EVENT_PULL_REQUEST_HEAD_LABEL: vivodi:fix_flush
    GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FULL_NAME: vivodi/Flexget
    GITHUB_REPOSITORY: Flexget/Flexget
==> Fork detected, setting branch to vivodi:fix_flush
Run if [ -z "$CC_SHA" ];
  if [ -z "$CC_SHA" ];
  then
    CC_SHA="$GITHUB_EVENT_PULL_REQUEST_HEAD_SHA"
  fi
  if [ -z "$CC_PR" ] && [ "${GITHUB_EVENT_NAME}" == "pull_request_target" ];
  then
    CC_PR="$GITHUB_EVENT_NUMBER"
  fi
  
  echo "CC_SHA=$CC_SHA" >> "$GITHUB_ENV"
  echo "CC_PR=$CC_PR" >> "$GITHUB_ENV"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
    CC_FORK: true
    TOKENLESS: vivodi:fix_flush
    CC_BRANCH: vivodi:fix_flush
    CC_PR: 
    CC_SHA: 
    GITHUB_EVENT_NAME: pull_request
    GITHUB_EVENT_NUMBER: 4367
    GITHUB_EVENT_PULL_REQUEST_HEAD_SHA: 4ce231c77f7[5](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:5)8cb4af29b90357b4a52b9c1e9c22
Run ${GITHUB_ACTION_PATH}/dist/codecov.sh
  ${GITHUB_ACTION_PATH}/dist/codecov.sh
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    UV_VERSION: 0.7.8
    UV_PYTHON: 3.13
    UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
    CC_FORK: true
    TOKENLESS: vivodi:fix_flush
    CC_BRANCH: vivodi:fix_flush
    CC_SHA: 4ce231c77f758cb4af29b90357b4a52b9c1e9c22
    CC_PR: 
    CC_BASE_SHA: 
    CC_BINARY: 
    CC_BUILD: 
    CC_BUILD_URL: 
    CC_CODE: 
    CC_DIR: 
    CC_DISABLE_FILE_FIXES: false
    CC_DISABLE_SEARCH: false
    CC_DISABLE_TELEM: false
    CC_DRY_RUN: false
    CC_ENTERPRISE_URL: 
    CC_ENV: 
    CC_EXCLUDES: 
    CC_FAIL_ON_ERROR: true
    CC_FILES: 
    CC_FLAGS: 
    CC_FORCE: 
    CC_GCOV_ARGS: 
    CC_GCOV_EXECUTABLE: gcov
    CC_GCOV_IGNORE: 
    CC_GCOV_INCLUDE: 
    CC_GIT_SERVICE: github
    CC_HANDLE_NO_REPORTS_FOUND: false
    CC_JOB_CODE: 
    CC_LEGACY: false
    CC_NAME: ubuntu-latest-3.13
    CC_NETWORK_FILTER: 
    CC_NETWORK_PREFIX: 
    CC_NETWORK_ROOT_FOLDER: 
    CC_OS: 
    CC_PARENT_SHA: 
    CC_PLUGINS: 
    CC_RECURSE_SUBMODULES: false
    CC_REPORT_TYPE: 
    CC_RUN_CMD: upload-coverage
    CC_SERVICE: github
    CC_SKIP_VALIDATION: false
    CC_SLUG: 
    CC_SWIFT_PROJECT: 
    CC_USE_PYPI: false
    CC_VERBOSE: false
    CC_VERSION: latest
    CC_YML_PATH: 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/
                            Wrapper-0.2.1
                                  
==> Detected linux
 -> Downloading https://cli.codecov.io/latest/linux/codecov
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 10.[6](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:6)M  100 10.6M    0     0   131M      0 --:--:-- --:--:-- --:--:--  131M
==> Finishing downloading linux:latest
      Version: v11.0.2
 
gpg: key 806BB28AED[7](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:7)79869: "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
==> Verifying GPG signature integrity
 -> Downloading https://cli.codecov.io/latest/linux/codecov.SHA256SUM
 -> Downloading https://cli.codecov.io/latest/linux/codecov.SHA256SUM.sig
 
gpg: Signature made Thu May 29 21:23:31 2025 UTC
gpg:                using RSA key 27034E7FDB[8](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:8)50E0BBC2C62FF806BB28AED77[9](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:9)869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C  62FF 806B B28A ED77 9869
codecov: OK
==> CLI integrity verified

 -> Token of length 0 detected
==> Running upload-coverage
      ./codecov  upload-coverage --fail-on-error --git-service github --sha 4ce231c77f758cb4af29b90357b4a52b9c1e9c22 --branch vivodi:fix_flush --gcov-executable gcov --name ubuntu-latest-3.13
info - 2025-05-30 [11](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:11):59:46,454 -- ci service found: github-actions
warning - 2025-05-30 11:59:46,495 -- xcrun is not installed or can't be found.
warning - 2025-05-30 11:59:46,666 -- No gcov data found.
warning - 2025-05-30 11:59:46,667 -- coverage.py is not installed or can't be found.
info - [20](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:20)25-05-30 11:59:46,705 -- Found 1 coverage files to report
info - 2025-05-30 11:59:46,705 -- > /home/runner/work/Flexget/Flexget/coverage.xml
info - 2025-05-30 11:59:47,339 -- Your upload is now processing. When finished, results will be available at: https://app.codecov.io/github/Flexget/Flexget/commit/4ce231c77f758cb4af29b90357b4a52b9c1e9c[22](https://github.com/Flexget/Flexget/actions/runs/15346195734/job/43182837366#step:7:22)
info - 2025-05-30 11:59:47,868 -- Process Upload complete

vivodi avatar May 30 '25 12:05 vivodi

@vivodi I have a suspicion this is happening because the PR is in a draft state.

thomasrockhu-codecov avatar May 31 '25 00:05 thomasrockhu-codecov

@vivodi I have a suspicion this is happening because the PR is in a draft state.

Previously, this PR was not marked as a draft because I was concerned that a collaborator might merge it before you @thomasrockhu-codecov had a chance to analyze the issue.

You can verify this by checking the event timestamp of the PR — I marked the PR as draft some time after the coverage had already been uploaded to Codecov. Alternatively, you can check another PR with the same issue that was never marked as draft.

Another non-draft PR in this repository exhibits the exact same issue: https://github.com/Flexget/Flexget/pull/4413

This issue might in fact be a duplicate of #1825 [BUG] Can't get coverage report. However, since the description in #1825 is not very detailed, I submitted this issue to provide more context.

vivodi avatar May 31 '25 05:05 vivodi

Based on my observation, this issue only occurs with pull requests from forked repositories, but not with those from different branches within the same repository. In other words, repo1:master <- repo1:feature works fine, but repo1:master <- repo2:feature causes the problem.

vivodi avatar Jun 01 '25 09:06 vivodi

Please refer to the two draft pull requests in the aiohttp repository. Codecov correctly created comments for both while they were still in draft status. Even if they have since been converted to non-draft PRs, the event timestamps clearly indicate that the comments were made while the PRs were still drafts.

  • https://github.com/aio-libs/aiohttp/pull/11122
  • https://github.com/aio-libs/aiohttp/pull/11121

This demonstrates that the draft status of a PR has no effect on whether Codecov creates a comment. As a member of the Codecov team, you should be able to confirm the expected behavior of your product.

vivodi avatar Jun 02 '25 17:06 vivodi

You can submit a PR to https://github.com/Flexget/Flexget or https://github.com/maplibre/maplibre-gl-js, and you’ll find that Codecov does not post a comment. If you open the Codecov UI and check under “Pulls”, you’ll see that your PR is marked as “missing head commit.” However, PRs from different branches within the same repository do not have this issue — the problem only occurs with PRs from forked repositories.

@thomasrockhu-codecov Why did you unassign yourself? Could you assign someone else to help me with this issue? Also, @HareIM in https://github.com/codecov/codecov-action/issues/1825 has the same problem as I do.

vivodi avatar Jun 21 '25 01:06 vivodi

Image I am having a similar issue, its not understanding the coverage for some reason...

mrz1836 avatar Jul 27 '25 00:07 mrz1836

Image I am having a similar issue, its not understanding the coverage for some reason...

Same here, the main branch definitely had multiple verified coverage uploads from GH, but still no message in PR, and the same error about Missing Head Commit is shown on Codecov website (same as for OP).

tsinis avatar Jul 27 '25 10:07 tsinis

I also have the same issue. Based on @vivodi 's comment here, I think our issue might be different than this one and #1825, so maybe we should open a new issue? I definitely have problems both in the main branch, and PRs in the same repo, not a different fork.

oir avatar Jul 27 '25 19:07 oir

Realizing we too are affected by this issue, it seems like a big deal in terms of making the product much less useful. Are there any updates on workarounds or the status here? We love codecov so this is a big bummer and hinders collaboration on our projects.

Same “missing head commit” despite successful uploads etc

Image

shamoon avatar Aug 21 '25 07:08 shamoon

Not sure if it's actually the right fix or not, but carryforward flags worked as a work-around for me.

flag_management:
  default_rules:
    carryforward: true # This keeps coverage from previous uploads
    statuses:
      - type: project
        target: auto
        threshold: 1%

mrz1836 avatar Aug 21 '25 11:08 mrz1836

Not sure if it's actually the right fix or not, but carryforward flags worked as a work-around for me.

flag_management:
  default_rules:
    carryforward: true # This keeps coverage from previous uploads
    statuses:
      - type: project
        target: auto
        threshold: 1%

This workaround doesn’t apply to me at all, since I don’t use flags. My configuration is as follows:

codecov:
  notify:
    after_n_builds: 15

coverage:
  status:
    patch:
      default:
        target: 100%
        informational: true
    project:
      default:
        target: auto

vivodi avatar Aug 21 '25 12:08 vivodi

Here is a (non-draft) PR that exactly illustrates this bug https://github.com/paperless-ngx/paperless-ngx/pull/10539

  • Action runs fine and coverage is uploaded: https://github.com/paperless-ngx/paperless-ngx/actions/runs/17077459188/job/48422530139 —> https://app.codecov.io/github/paperless-ngx/paperless-ngx/commit/58f5151ce5115b5d11b62e4425ed0c601828d22c info - 2025-08-19 17:50:58,342 -- Process Upload complete
  • No comment on the PR
  • Pull on codecov shows “missing head commit” https://app.codecov.io/github/paperless-ngx/paperless-ngx/pull/10539

This issue has been open for almost 2 months, without any updates from the team. @thomasrockhu-codecov @adrianviquez sorry to bother, we understand you are busy, but an update would be very appreciated.

Seems the same as #1852

shamoon avatar Aug 21 '25 16:08 shamoon

Working on a fix that I hope to deploy tomorrow morning

thomasrockhu-codecov avatar Sep 03 '25 21:09 thomasrockhu-codecov

Working on a fix that I hope to deploy tomorrow morning

@thomasrockhu-codecov Using the latest codecov-action v5.5.1 did not resolve the issue — comments still don’t appear for PRs from forks, and the Codecov PR page shows “Missing Head Commit.”

PR: https://github.com/Flexget/Flexget/pull/4607 workflow run: https://github.com/Flexget/Flexget/actions/runs/17467985329/job/49608661847?pr=4607 Codecov PR page: https://app.codecov.io/gh/Flexget/Flexget/pull/4607

vivodi avatar Sep 04 '25 15:09 vivodi

@vivodi thanks for the data, investigating, I don't think it's an action thing so digging into our backend

thomasrockhu-codecov avatar Sep 04 '25 15:09 thomasrockhu-codecov

Tom has tasked me with getting to the bottom of this. I believe it's due to some weirdness in how the GitHub API is reporting if a commit is part of a PR or not. I will keep you updated.

drazisil-codecov avatar Sep 29 '25 15:09 drazisil-codecov

I used the solution mentioned in https://github.com/codecov/feedback/issues/789#issuecomment-3448221444 on the FlexGet repository (https://github.com/Flexget/Flexget/pull/4700), and all the problems were solved.

@thomasrockhu-codecov I think the codecov/test-results-action repository should be archived as soon as possible, and its README should state that it is deprecated to prevent others from encountering the same error as this PR due to using it.

vivodi avatar Oct 26 '25 12:10 vivodi

@vivodi loud and clear it should be deprecated, it's on my list to do tomorrow

thomasrockhu-codecov avatar Oct 26 '25 19:10 thomasrockhu-codecov