dask-kubernetes icon indicating copy to clipboard operation
dask-kubernetes copied to clipboard

Try running tests in parallel

Open jacobtomlinson opened this issue 1 year ago • 1 comments

The CI is taking nearly 30 minutes at the moment, experimenting with running tests in parallel to bring that time down.

Initial timings:

Suite Time (approx)
HelmCluster 5 mins
KubeCluster (classic) 28 mins
KubeCluster (operator) 21 mins

jacobtomlinson avatar Sep 12 '22 14:09 jacobtomlinson

Failures are happening because the first two tests in each process are both trying to create the session fixtures and are clobbering each other.

We lave the k8s cluster running after the tests have run, so one solution could be to run a single test without parallelism first which would cause the cluster to be created. Then run all the tests with some parallelism which would skip the cluster setup.

jacobtomlinson avatar Sep 13 '22 11:09 jacobtomlinson