azure-sdk-for-c
azure-sdk-for-c copied to clipboard
Azure
CI incorrectly reports test coverage being too low (0%) when build fails
From https://github.com/Azure/azure-sdk-for-c/pull/550 (particularly https://github.com/Azure/azure-sdk-for-c/pull/550/commits/43243fb788e5c760c366d90bbe317873bea8d959)
https://github.com/Azure/azure-sdk-for-c/pull/550/checks?check_run_id=578163302
https://dev.azure.com/azure-sdk/public/_build/results?buildId=352744&view=logs&jobId=5e607c67-84c7-5489-c4f9-bbc2a3f1af84&j=c405da93-63bc-52c4-660e-dc11361a3bc8&t=8f16da27-6ab0-5581-5587-361f20664c13
Starting: Check line coverage
==============================================================================
Task : Build Quality Checks
Description : Breaks a build based on quality metrics like number of warnings or code coverage.
Version : 6.4.4
Author : Microsoft Premier Services
Help : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
==============================================================================
SystemVssConnection exists true
Using IdentifierJobResolver
Validating code coverage policy...
Waiting for code coverage data...
Waiting for code coverage data...
Waiting for code coverage data...
...
...
Waiting for code coverage data...
Waiting for code coverage data...
[WARNING] Unable to get code coverage data within the maximum wait time.
##[warning]Unable to get code coverage data within the maximum wait time.
Total line: 0
Covered line: 0
Code Coverage (%): 0
[ERROR] The code coverage value (0%, 0 line) is lower than the minimum value (85%)!
##[error]The code coverage value (0%, 0 line) is lower than the minimum value (85%)!
##[error]At least one build quality policy was violated. See Build Quality Checks section for more details.
(node:6395) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'sort' of null
Finishing: Check line coverage
cc @danieljurek
Looks like the annotation here shows the most recent failure. In this particular case the build failed (so no tests ran and no coverage was produced). Need to find a way to run coverage gates if and only if coverage information is available.
@danieljurek don't we use the success() condition for this? so only if build and test pass, then we run the coverage step
Need to find a way to run coverage gates if and only if coverage information is available.
@danieljurek, agreed. Seems like a conditioning fix in the pipeline.
