speed up testing

[Mikejmnez]

• OceanSpy version: most recent

Description

Running pytest -v takes ~ 6 minutes (laptop). The test file oceanspy/tests/test_open_oceandataset.pystalls for a very long there. Sometimes in the past I have had to kill the testing. I ran tests without the file as follows

pytest -v --ignore=oceanspy/tests/tests_opening_saving.py

and it took about 4 minutes to finish through. Better but not ideal...

Approaches:

• Refactor testing
• Add parallelism

What I Did

Add parallelism to the testing via pytest-xdist. pip Install it and then run:

pytest -v -n 4

It took consistently a little over 1 minute to be done with all testing (N=10)!!!. However, sometimes pytest would err before testing even began during collection the 4 workers

collecting: 0/4 workers

error was not very helpful. Trying again would always work. Not sure why... I think refactor is still needed, but this was a huge improvements.

Side note on slowest test files

Testing spit out a very interesting RunTimeError with the slow testing file :

oceanspy/tests/test_open_oceandataset.py::test_opening_and_saving[xarray-./oceanspy/tests/Data/catalog_xarray.yaml] - RuntimeError: only endian='native' allowed for NETCDF3 files, got 'big' (variable 'Tair', group '/')

Groups is part of NetCDF4 and not of netCDF3. Perhaps the reason of why that test takes a very long time?