Add additional config to skip violation check when the changes are less than a threshold

[praveen-kumar85]

When there are fewer changes, very often we don't cover all the possible instructions for these changes in the test cases. And in such cases, we should have the option to avoid the violation error by defining the threshold for fewer changes (number of instructions, lines, branches, etc.)

[SurpSG]

Hi @praveen-kumar85

Workaround

I solved the issue you discribed in the next way:

1. Declare a property in pom.xml:

...

    <properties>
        <failOnDiffCoverageViolation>true</failOnDiffCoverageViolation> <!-- fail by default -->
    </properties>

...

2. Setup the plugin to use the property:

            <plugin>
                <groupId>com.github.surpsg</groupId>
                <artifactId>diff-coverage-maven-plugin</artifactId>
                <version>0.3.2</version>
                <configuration>
                    ...
                    <violations>
                        <failOnViolation>${failOnDiffCoverageViolation}</failOnViolation>  <!-- consume the property -->
                        <minCoverage>0.9</minCoverage>
                    </violations>
                </configuration>
                ...
            </plugin>

3. Then invoke the plugin with overriding property value

mvn install -DfailOnDiffCoverageViolation=false

---

CI usage

Github Actions

Suppose you are using Github Actions:

1. Create a custom label (let's say ignore-diff-coverage).
2. Update your github action:

      - name: Build with Maven
        run: mvnw install -DfailOnDiffCoverageViolation=${{ !contains(github.event.pull_request.labels.*.name, 'ignore-diff-coverage') }}

OR

# in this case failOnDiffCoverageViolation maven property is not used for passing to maven 
# but it still would be handy for local run

      - name: Build with Maven
        continue-on-error: ${{ contains(github.event.pull_request.labels.*.name, 'ignore-diff-coverage') }} 
        run: mvnw install

3. Apply the label to your pull request that requires skipping of diff coverage check

Another CIs

You can apply the similar approach to Gitlab or Jenkins piplines for sure. Also, I belive you can use the approach for CIs like Travis or Circle.

Real example

• Pull request: https://github.com/SurpSG/diff-coverage-maven-plugin/pull/41 with label ignore-diff-coverage
• Logs where we can see that diff coverage was suppressed but by default the plugin fails a build if coverage violated.

Summary

Advantuges:

• You intetionally suppress diff coverage failure. Small change in code doesn't mean that the coverage check should be omitted. So you can decide whether ignore diff coverage check per particular PR.

Disadvantuges:

• You must manually suppress diff coverage failure if required 🙃

Yeah, it depends on your strategy.

[SurpSG]

If the approach above doesn't suitable for you then await few days(up to 2 weeks) while I implement the feature you ask.

[praveen-kumar85]

@SurpSG thanks a lot for the response. This is quite useful for manually adjusting the failure setting without having to touch the code. For our current use cases, it is good enough. But still, I think we should have a configurable parameter to skip the violations just the way how sonar does it.

[SurpSG]

@praveen-kumar85

Going to implement the feature you requested. Here example how it could be configured:

https://github.com/SurpSG/diff-coverage-maven-plugin/pull/50/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R62

I would be pleased to receive feedback