xdmod
xdmod copied to clipboard
Changes that add and integrate code coverage support into XDMoD's CI pipeline
NOTE: this PR takes the place of https://github.com/ubccr/xdmod/pull/1350
All of the functional changes are the work of Zi-3 all that I've (@ryanrath) done is clean things up a little.
Description
- Added four new scripts responsible for installing & configuring XDebug, generating the code coverage data from php file execution, and generating the final reports.
- Updated
shippable.yml
so that XDebug is installed / Code Coverage is only generated during the nightly test build. - Updated numerous UI tests to work more reliably when XDebug is installed ( i.e. their timeouts were increased )
- Updated a couple of UI test selectors to minimize the effects of latency on particular tests.
- Added a new webdriverhelper function called
clickSelectorAndWaitForMask
as this was needed for a test or two. - Updated the timeouts for a few other webdriverhelper functions.
Motivation and Context
Adding code coverage will allow us to get a general sense of where we are lacking test code & possibly where we have unused code thus giving us a few more tools to help streamline / increase the reliability of XDMoD.
Tests performed
All the automated tests for an install / upgrade with & without XDebug installed.
Checklist:
- [X] The pull request description is suitable for a Changelog entry
- [X] The milestone is set correctly on the pull request
- [X] The appropriate labels have been added to the pull request
I force triggered the code coverage generation in the last commit to make sure that it works. Before I set it back I just wanted to document the code coverage artifacts from each build:
- CentOS7 Upgrade: https://output.circle-artifacts.com/output/job/71e67d9a-19a5-4b42-b468-8ae812b1d269/artifacts/0/~/code_coverage/report.tgz
- CentOS7 Fresh Install: https://output.circle-artifacts.com/output/job/0a5c7d5f-d5d6-48d6-9bc6-4eb588b533a3/artifacts/0/~/code_coverage/report.tgz
- CentOS8: https://output.circle-artifacts.com/output/job/b6b7148c-b54e-4d04-befd-702c23d1ead5/artifacts/0/~/code_coverage/report.tgz
- Rocky 8: https://output.circle-artifacts.com/output/job/d13f18c3-15c9-4a31-885e-9264dd5169b5/artifacts/0/~/code_coverage/report.tgz