RFC6455 icon indicating copy to clipboard operation
RFC6455 copied to clipboard

Investigate concurrent testing

Open cboden opened this issue 2 years ago • 1 comments

With the Autobahn test suites now running on different ports for client/server we could investigate running them concurrently in GitHub Actions. This would reduce the number of jobs in half and possibly run the whole suite faster.

cboden avatar Dec 10 '21 16:12 cboden

I think you could split this into three separate jobs:

  • Autobahn Test Suite (Client) - The current sh tests/ab/run_ab_tests.sh step and then vendor/bin/phpunit tests/AbResultsTest.php for the "client" environment (or if the PHPUnit config is tweaked a bit, make this test a separate suite then it's a vendor/bin/phpunit --testsuite="Autobahn" type of command); in theory, this probably only needs to run on the latest supported PHP version (and maybe the oldest too if you want a build explicitly running with the lower dependencies for this one)
  • Autobahn Test Suite (Server) - Same as above, but the "server" environment instead
  • Unit Tests - The current vendor/bin/phpunit step, excluding the AbResultsTest; this one should have a run on every supported PHP version, and generally it's also a good idea to have a build in the matrix using Composer's --prefer-lowest option which helps validate the lower bound of the dependencies

It doesn't really speed a single build up all that much just because of how long it takes the Autobahn suites to run, but it would reduce the number of builds running those long test suites without sacrificing any code coverage.

mbabker avatar Dec 12 '21 01:12 mbabker