awslabs
Reduce time for integration test
Come up with a way so that CI prints the running time for each tests. Find what integration tests are running longer and optimize them to make them run fast. Current tests are taking 1:30 minutes, that needs to get faster.
I have done some optimization. Current time is 38 minutes for tests to run. Some unit tests are taking more than 5 minutes. Reports can be found in CI logs.
Xgboost integ test is disabled and there are couple of others TF test which are disable.
Time taken by unit test needs to be optimized 23.35s call tests/analysis/trials/test_s3.py::test_s3 22.24s call tests/analysis/trials/test_s3.py::test_multiple_s3_trials 9.83s call tests/analysis/trials/test_refresh.py::test_refresh_with_range_s3 8.48s call tests/analysis/trials/test_refresh.py::test_refresh_s3 7.83s call tests/analysis/trials/test_refresh.py::test_no_refresh_s3 5.14s call tests/analysis/tensors/test_refresh.py::test_refresh_tensors 4.85s call tests/analysis/trials/test_create.py::test_creation_s3
209.22s call tests/core/test_index_reader.py::test_fetch_tensor_with_missing_event_file_but_next_event_file_present_incomplete_job
8.26s call tests/core/test_hook_save_scalar.py::test_tf_save_scalar[save_config1-collection0]
8.00s call tests/core/test_hook_save_scalar.py::test_tf_save_scalar[save_config0-collection0]
6.95s call tests/core/test_hook_save_scalar.py::test_tf_save_scalar[save_config1-collection1]
6.66s call tests/core/test_hook_save_scalar.py::test_tf_save_scalar[save_config0-collection1]
3.66s call tests/core/test_hook_save_scalar.py::test_pytorch_save_scalar[True-save_config1-collection0]
3.56s call tests/core/test_hook_save_scalar.py::test_pytorch_save_scalar[False-save_config1-collection0]
3.19s call tests/core/test_hook_save_scalar.py::test_pytorch_save_scalar[True-save_config0-collection0]
3.05s call tests/core/test_hook_save_scalar.py::test_pytorch_save_scalar[False-save_config0-collection0]
2.77s call tests/core/test_hook_save_scalar.py::test_mxnet_save_scalar[True-save_config1-collection0] 2.74s call tests/core/test_hook_save_scalar.py::test_mxnet_save_scalar[False-save_config1-collection0]
2.39s call tests/core/test_hook_save_scalar.py::test_mxnet_save_scalar[True-save_config0-collection0]