openmct icon indicating copy to clipboard operation
openmct copied to clipboard

[CI] Configure circleci e2e test execution to leverage e2e re-runfailures and intelligent test sharding

Open unlikelyzero opened this issue 2 years ago • 0 comments

Summary

Circle CI has the ability the intelligently shard e2e tests so that they're always running as fast as possible when taking into account historical runtime. At the same time, Circle CI has added rerunfailed tests: https://circleci.com/docs/rerun-failed-tests/#configure-a-job-running-playwright-tests

We chose not to implement that because we didn't know how that would affect our use of github actions/workflows and internal bamboo runs.

At this time, there is not technical reason not to invest in this and we should see speed improvements instantly. It also is a better parallelization model for us given the recent playwright changes to shard by test file and not by testcase/testsuite.

  • [ ] Modify CircleCI Config to include historical test runtime
  • [ ] Make further changes to support https://circleci.com/docs/rerun-failed-tests/#configure-a-job-running-playwright-tests if necessary

unlikelyzero avatar Aug 16 '23 19:08 unlikelyzero