openmct
openmct copied to clipboard
[CI] Configure circleci e2e test execution to leverage e2e re-runfailures and intelligent test sharding
Summary
Circle CI has the ability the intelligently shard e2e tests so that they're always running as fast as possible when taking into account historical runtime. At the same time, Circle CI has added rerunfailed tests: https://circleci.com/docs/rerun-failed-tests/#configure-a-job-running-playwright-tests
We chose not to implement that because we didn't know how that would affect our use of github actions/workflows and internal bamboo runs.
At this time, there is not technical reason not to invest in this and we should see speed improvements instantly. It also is a better parallelization model for us given the recent playwright changes to shard by test file and not by testcase/testsuite.
- [ ] Modify CircleCI Config to include historical test runtime
- [ ] Make further changes to support https://circleci.com/docs/rerun-failed-tests/#configure-a-job-running-playwright-tests if necessary