rnp icon indicating copy to clipboard operation
rnp copied to clipboard

Published 20 hours ago • verified publisher icon rnpgp

Optimize time of tests

Open ribose-jeffreylau opened this issue 3 years ago • 1 comments

Description

Tests on GitHub Actions take a long while to complete, despite individual tests not taking too long. This could be due to the sheer number of tests.

Investigate whether the number of tests can be reduced, by cutting down on the number of build combinations / etc., without impacting the confidence / coverage of the tests.

Expected Behavior

The entire test suite takes within 1 hour to complete. (I actually don't have a specific target in mind...)

Actual Behavior

The entire test suite takes > about 1.5hrs to complete.

ribose-jeffreylau avatar Nov 12 '21 04:11 ribose-jeffreylau

I see the following duplications:

  • redundant coverage tests. With the same backend coverage report should be the same independent on OS. I.e. run only one coverage job on centos 8 with Botan backend and OpenSSL backend. In addition, Codecov doesn't seem to work at all :)
  • redundant sanitizer jobs. Shouldn't it be enough to rung single sanitize job for Centos 8 and for macOS?
  • redundant GPG beta and 2.3.1 jobs. Shouldn't we run that only once on Centos 8 or Ubuntu, as we are checking only data format compatibility, not the library compilation issues or whatever else?

ni4 avatar Nov 12 '21 10:11 ni4