Extract Parallel Tests into Separate Github CI Action

[chukarsten]

As an EvalML dev, I would like parallel CI tests to be put into a separate Github Action CI check so that development can carry on if they prove flaky.

Parallel tests with DaskEngine and CFEngine have proven less than resilient to dependency changes. Until the specific cause of this flakiness can be determined, the tests for parallel behavior should be extracted and put into a separate CI check that is allowed to flake and fail periodically.

Here are the locations of "git-test-parallel"

[LunaCodeDemon]

I don't know EvalML, but i know pytest. Did you guys add a pip install --r requirements.txt into workflows that need dependencies?

[chukarsten]

Nope. Our github CI yamls install the deps via our Makefile and, for our basic environment, installs from the directory with pip install -e . -q. We've been moving away from the requirements.txt and over to setup.cfg