InnerEye-DeepLearning icon indicating copy to clipboard operation
InnerEye-DeepLearning copied to clipboard

Make unit tests faster

Open dumbledad opened this issue 3 years ago • 1 comments

At present running all the unit tests in WSL on my laptop takes 42 minutes. That is too long for me to run all the tests locally before pushing any changes I have made. The unit tests would be more useful if they ran so fast that I would routinely run them all locally while I work on a PR.

The tests that run for more than a minute (in WSL on my XPS15) are:

394.35s call Tests/AfterTraining/test_after_training.py::test_submit_for_inference[True] 207.80s call Tests/SSL/test_ssl_containers.py::test_innereye_ssl_container_cifar10_resnet_simclr 127.20s call Tests/SSL/test_data_modules.py::test_innereye_vision_module 123.78s call Tests/ML/utils/test_checkpoint_handling.py::test_get_best_checkpoint_single_run 62.47s call Tests/ML/utils/test_checkpoint_handling.py::test_get_recovery_path_train_single_run 62.26s call Tests/ML/utils/test_checkpoint_handling.py::test_download_checkpoints_from_single_run 62.14s call Tests/ML/utils/test_checkpoint_handling.py::test_get_checkpoints_to_test_single_run 61.99s call Tests/ML/test_download_upload.py::test_download_recovery_single_run

AB#4166

dumbledad avatar Jun 28 '21 13:06 dumbledad

Note that 11 fail locally which pass in our Azure build pipeline:

FAILED Tests/AfterTraining/test_after_training.py::test_submit_for_inference[False] - azureml.exceptions._azureml_exc... FAILED Tests/AfterTraining/test_after_training.py::test_submit_for_inference[True] - azureml.exceptions._azureml_exce... FAILED Tests/AfterTraining/test_after_training.py::test_recovery_on_2_nodes - azureml.exceptions._azureml_exception.S... FAILED Tests/ML/test_download_upload.py::test_download_azureml_dataset - NotImplementedError: Linux distribution ubun... FAILED Tests/ML/test_lightning_containers.py::test_create_fastmri_container - ModuleNotFoundError: No module named 'f... FAILED Tests/ML/test_lightning_containers.py::test_run_fastmri_container - ValueError: Model name FastMriOnRandomData... FAILED Tests/ML/visualizers/test_plot_cross_validation.py::test_add_comparison_data - Exception: Run refs_pull_444_me... FAILED Tests/ML/visualizers/test_visualize_patches.py::test_visualize_patch_sampling[True] - AssertionError: Image si... FAILED Tests/ML/visualizers/test_visualize_patches.py::test_visualize_patch_sampling_2d - AssertionError: Image sizes... FAILED Tests/SSL/test_data_modules.py::test_innereye_vision_module - NameError: name 'cv2' is not defined FAILED Tests/SSL/test_ssl_containers.py::test_innereye_ssl_container_cifar10_resnet_simclr - NameError: name 'cv2' is...

dumbledad avatar Jun 28 '21 13:06 dumbledad