xdmod icon indicating copy to clipboard operation
xdmod copied to clipboard

Changes that add and integrate code coverage support into XDMoD's CI pipeline

Open ryanrath opened this issue 3 years ago • 1 comments

NOTE: this PR takes the place of https://github.com/ubccr/xdmod/pull/1350

All of the functional changes are the work of Zi-3 all that I've (@ryanrath) done is clean things up a little.

Description

  • Added four new scripts responsible for installing & configuring XDebug, generating the code coverage data from php file execution, and generating the final reports.
  • Updated shippable.yml so that XDebug is installed / Code Coverage is only generated during the nightly test build.
  • Updated numerous UI tests to work more reliably when XDebug is installed ( i.e. their timeouts were increased )
  • Updated a couple of UI test selectors to minimize the effects of latency on particular tests.
  • Added a new webdriverhelper function called clickSelectorAndWaitForMask as this was needed for a test or two.
  • Updated the timeouts for a few other webdriverhelper functions.

Motivation and Context

Adding code coverage will allow us to get a general sense of where we are lacking test code & possibly where we have unused code thus giving us a few more tools to help streamline / increase the reliability of XDMoD.

Tests performed

All the automated tests for an install / upgrade with & without XDebug installed.

Checklist:

  • [X] The pull request description is suitable for a Changelog entry
  • [X] The milestone is set correctly on the pull request
  • [X] The appropriate labels have been added to the pull request

ryanrath avatar Sep 09 '20 18:09 ryanrath

I force triggered the code coverage generation in the last commit to make sure that it works. Before I set it back I just wanted to document the code coverage artifacts from each build:

  • CentOS7 Upgrade: https://output.circle-artifacts.com/output/job/71e67d9a-19a5-4b42-b468-8ae812b1d269/artifacts/0/~/code_coverage/report.tgz
  • CentOS7 Fresh Install: https://output.circle-artifacts.com/output/job/0a5c7d5f-d5d6-48d6-9bc6-4eb588b533a3/artifacts/0/~/code_coverage/report.tgz
  • CentOS8: https://output.circle-artifacts.com/output/job/b6b7148c-b54e-4d04-befd-702c23d1ead5/artifacts/0/~/code_coverage/report.tgz
  • Rocky 8: https://output.circle-artifacts.com/output/job/d13f18c3-15c9-4a31-885e-9264dd5169b5/artifacts/0/~/code_coverage/report.tgz

ryanrath avatar Sep 02 '22 20:09 ryanrath