code-coverage-api-plugin icon indicating copy to clipboard operation
code-coverage-api-plugin copied to clipboard

failBuildIfCoverageDecreasedInChangeRequest should not fail if no code changed

Open fuzzybair opened this issue 4 years ago • 0 comments

Version report

Jenkins and plugins versions report:

Jenkins: 2.263.1
OS: Linux - 4.15.0-136-generic
---
sonar:2.12
blueocean-core-js:1.24.3
git:4.4.4
cobertura:1.16
[Many others as this is a shared server]
  • What Operating System are you using (both controller, and any agents involved in the problem)?
OS: Linux - 4.15.0-136-generic

Reproduction steps

We have several items that are not code related in our repository (e.g. documentation, images, etc.). when we have a pull request that is strictly non code the Pull Request can fail. The failure seems to be related to a rounding difference between the stored and calculated values. Possible solutions:

  • Add a level of tolerance to failBuildIfCoverageDecreasedInChangeRequest to account for the rounding.
  • Always round down before storing the value
  • Use the same precision with stored and calculated.

Results

Here is our configuration for the code coverage.

      publishCoverage adapters: [coberturaAdapter(mergeToOneReport: false, path: 'codeCoverageReports/Cobertura.xml')],
        calculateDiffForChangeRequests: true, failNoReports: true, failUnhealthy: false, failUnstable: true, failBuildIfCoverageDecreasedInChangeRequest: true,
        globalThresholds: [[thresholdTarget: 'Line', unhealthyThreshold: 85.0, unstableThreshold: 80.0]], sourceFileResolver: sourceFiles('STORE_LAST_BUILD') 

Expected result:

failBuildIfCoverageDecreasedInChangeRequest should not fail if there are not any associated code changes.

Actual result:

We have observed values indicated coverage changed on these pull request between -0.004% and +0.004% if the rounding rounds down from the calculated value the request will show an increase in coverage. But if the request follows after a merge that rounded up it will show a decrease in coverage, and therefore fail the check.

fuzzybair avatar Apr 01 '21 00:04 fuzzybair