codecov-action icon indicating copy to clipboard operation
codecov-action copied to clipboard

Setting threshold

Open serialbandicoot opened this issue 4 years ago • 6 comments

I'm trying to set threshold, I've added as below, however it doesn't seem to make any difference. Can you please point me in the direction of the docs.

with:
     token: ${{ secrets.CODECOV_TOKEN }}
     files: coverage.xml
     env_vars: OS,PYTHON
     fail_ci_if_error: true
     threshold: 2%

serialbandicoot avatar Oct 11 '21 12:10 serialbandicoot

I'm getting a warning with the following list of acceptable values. So can I assume this is not the way to approach Threshold and guess I must provide another mechanism. Any pointers would be super-helpful!

run (ubuntu-latest)Unexpected input(s)
 'threshold', valid inputs are ['token', 'files', 'directory', 'flags', 'commit_parent', 'dry_run', 'env_vars', 'fail_ci_if_error', 'file', 'functionalities', 'move_coverage_to_trash', 'name', 'override_branch', 'override_build', 'override_commit', 'override_pr', 'override_tag', 'os', 'root_dir', 'slug', 'url', 'verbose', 'version', 'working-directory']

--

serialbandicoot avatar Oct 13 '21 10:10 serialbandicoot

We have codecov.yml file in repo root, but looks like it is being ignored completely for threshold. CI fails even with 0.01% of coverage diff. I am unsure if this is a side-effect of parallel tests on matrix of OS/Python versions.

Our config file -- https://github.com/abhinavsingh/proxy.py/blob/develop/codecov.yml Example CI failure which seems to ignore the threshold settings in the config -- https://github.com/abhinavsingh/proxy.py/pull/897

This seemed to work fine with v1 afaik

abhinavsingh avatar Dec 21 '21 20:12 abhinavsingh

Having this same issue... config seems to be ignored but can't add to action...

yaboi avatar Jan 14 '22 21:01 yaboi

The threshold option is not supported by the action—it will do nothing:

https://github.com/codecov/codecov-action/blob/e3c560433a6cc60aec8812599b7844a7b4fa0d71/action.yml#L4-L91

I don't see the way to point to the codecov.yml file either.

Can this action determine the root-level codecov.yml configuration file? Can we mention this in the README?

kettanaito avatar Apr 11 '22 11:04 kettanaito

There no threshold?

mataide avatar Aug 04 '22 03:08 mataide

Is there anyway to point to thecodecov.yml file?

trajan0x avatar Aug 10 '22 15:08 trajan0x

@thomasrockhu-codecov @mitchell-codecov can you please advise us on how to proceed with this issue ?

arkodg avatar Sep 27 '22 23:09 arkodg

Hi all,

The codecov.yml file holds configuration pieces like threshold. At the repository level, it can be named

  • codecov.yml (preferred)
  • codecov.yaml
  • .codecov.yml
  • .codecov.yaml

and must be directly in one of these directories

  • / (root)
  • 'dev/`
  • .github/

thomasrockhu-codecov avatar Sep 28 '22 17:09 thomasrockhu-codecov

great thanks for the pointer @thomasrockhu-codecov, this should unblock us. If you gave us some hints on how to enhance the GHA, one of us from the community could help out !

arkodg avatar Sep 28 '22 20:09 arkodg

After setting threshold the wrong way top level, I saw in the docs, that it is a value nested quite deep into the configuration file (as already explained perfectly prior):

coverage:
  status:
    project:
      default:
        threshold: 1%

Or see the pull request we used in our project.

noxan avatar Nov 28 '22 23:11 noxan

Hi all,

The codecov.yml file holds configuration pieces like threshold. At the repository level, it can be named

  • codecov.yml (preferred)
  • codecov.yaml
  • .codecov.yml
  • .codecov.yaml

and must be directly in one of these directories

  • / (root)
  • 'dev/`
  • .github/

After setting threshold in .github/codecov.yml,

coverage:
  status:
    project:
      default:
        # basic
        target: 60%
        threshold: 5%
        if_ci_failed: error

The same configuration in my repository, it works.
image

but in remote repository, it not work, CI success. Can you give me some advises or information?

ethanhanjiahao avatar Dec 10 '22 05:12 ethanhanjiahao

@ethanhanjiahao does your remote repo also have the Codecov yaml file?

thomasrockhu-codecov avatar Dec 12 '22 21:12 thomasrockhu-codecov

hi @thomasrockhu-codecov, answering on behalf of @ethanhanjiahao, it does, and its here, yet we saw in this PR that a drop in the coverage below threshold didn't fail the CI

arkodg avatar Dec 19 '22 21:12 arkodg

@arkodg please add the Codecov GitHub app to the repo. I suspect that's why it isn't getting picked up properly.

If you are still having an issue, please open an issue on our community boards

thomasrockhu-codecov avatar Mar 01 '23 19:03 thomasrockhu-codecov