oneDPL icon indicating copy to clipboard operation
oneDPL copied to clipboard

[Tests] Shorten sycl_iterator and permutation_iterator tests

Open danhoeflinger opened this issue 9 months ago • 6 comments

Adds infrastructure to be able to shorten some commonly used test infrastructure by

  1. allowing scaling of the max_n (100000) and small_n (16)
  2. allowing scaling of the step (3.14.15) multiplier

This also corrects a potential error in the search_20 test of sycl_iterator_find which for some value of n, could return the incorrect result due to a unintentional extension of the searched for list by the nature of the input data.

A few tests were scaled in this PR to have fewer iterations with different sizes:

sycl_iterator_find sycl_iterator_reduce sycl_iterator_scan sycl_iterator_sort permutation_iterator_parallel_merge permutation_iterator_parallel_stable_sort

Others were just given the opportunity for similar shortening.

These patterns should also be tested via their individual tests, independently from the sycl_iterator type or the permutation_iterator types. Also, the newly added input_data_sweep tests, provide much more coverage for input data processing to dpcpp backends.

danhoeflinger avatar May 01 '24 17:05 danhoeflinger

Note: This does not do anything to address time spent in JIT.
Its quite possible that a lot of the time of the test is spent in JIT compilation, and only removing kernel invocations or splitting the test can address this.

danhoeflinger avatar May 01 '24 18:05 danhoeflinger

Should we think about ability to save previous behavior when the macros TEST_LONG_RUN is defined?

SergeyKopienko avatar May 02 '24 07:05 SergeyKopienko

Should we think about ability to save previous behavior when the macros TEST_LONG_RUN is defined?

Yes, perhaps with a we can do a min / max with the scale factor of 1.0 when TEST_LONG_RUN is defined so that we don't shorten a test someone intentionally made longer with the scale factors. Its a good idea.

danhoeflinger avatar May 02 '24 12:05 danhoeflinger

Should we think about ability to save previous behavior when the macros TEST_LONG_RUN is defined?

I've now made it so when TEST_LONG_RUN is true, we undo any reduction in test cases from this. We allow scaling factors to still increase the number of tests in this case. None of the current code uses these scaling factors to increase the number of tests, but it is possible in the future, and we wouldn't want to reduce that for TEST_LONG_RUN.

danhoeflinger avatar May 09 '24 15:05 danhoeflinger

Actually... it looks like TEST_LONG_RUN=1 is used in the nightly testing, so it kind of defeats the purpose of this PR to turn off the savings in this case if our goal is to speed up the full suite testing.

danhoeflinger avatar May 09 '24 17:05 danhoeflinger

Actually... it looks like TEST_LONG_RUN=1 is used in the nightly testing, so it kind of defeats the purpose of this PR to turn off the savings in this case if our goal is to speed up the full suite testing.

I'm tempted to undo my TEST_LONG_RUN changes so we can take advantage of the savings in the nightly suite.

@timmiesmith @SergeyKopienko What are your thoughts?

danhoeflinger avatar May 13 '24 12:05 danhoeflinger

I'm tempted to undo my TEST_LONG_RUN changes so we can take advantage of the savings in the nightly suite.

I've reverted this change, because the point is to provide savings to the nightly suite. We can examine this further, perhaps we want some TEST_VERY_LONG_RUN option or something.

danhoeflinger avatar May 20 '24 13:05 danhoeflinger