NimbusML icon indicating copy to clipboard operation
NimbusML copied to clipboard

Remove pytest-cov test coverage report from validation builds.

Open montebhoover opened this issue 5 years ago • 1 comments

We currently run pytest-cov to get a test coverage report on all of our CI validation builds, but it seems to have errors in downloading plugins or parsing files in 10-25% of builds. (See https://dev.azure.com/aifx/public/_build/results?buildId=73&view=logs)

I like the idea of running a test coverage report on validation builds to ensure that new code has unit tests added, but the way we currently do this there is no such forcing function - the coverage report just gets silently written, and is in fact not even uploaded off the build agent (this is an oversight on my part that can be easily fixed).

Given the fact that pytest-cov causes so many build failures, I feel that the benefits are not worth the cost and I suggest we remove it from our build scripts. If we feel it is worth the effort, we could set up special builds specifically for producing coverage reports, or set aside time for debugging the pytest-cov failures.

montebhoover avatar Oct 25 '18 18:10 montebhoover

@pieths pls investigate if we can fix cov reports. If not then remove them

ganik avatar Oct 17 '19 14:10 ganik