dask-kubernetes
dask-kubernetes copied to clipboard
Try running tests in parallel
The CI is taking nearly 30 minutes at the moment, experimenting with running tests in parallel to bring that time down.
Initial timings:
Suite | Time (approx) |
---|---|
HelmCluster |
5 mins |
KubeCluster (classic) |
28 mins |
KubeCluster (operator) |
21 mins |
Failures are happening because the first two tests in each process are both trying to create the session fixtures and are clobbering each other.
We lave the k8s cluster running after the tests have run, so one solution could be to run a single test without parallelism first which would cause the cluster to be created. Then run all the tests with some parallelism which would skip the cluster setup.