llvm icon indicating copy to clipboard operation
llvm copied to clipboard

[SYCL][E2E] Functionality for splitting build and run of e2e tests

Open ayylol opened this issue 1 year ago • 1 comments

ayylol avatar Oct 16 '24 17:10 ayylol

Dry run of e2e test builds with no logic for setting triple or treating tests as unsupported: https://github.com/intel/llvm/actions/runs/11388738602

68 tests failed, so currently that should be the max amount of unsupported tests for building with the spir64 triple on Linux.

ayylol avatar Oct 17 '24 18:10 ayylol

Dry run of e2e test builds with no logic for setting triple or treating tests as unsupported: https://github.com/intel/llvm/actions/runs/11388738602

68 tests failed, so currently that should be the max amount of unsupported tests for building with the spir64 triple on Linux.

Latest dry run https://github.com/intel/llvm/actions/runs/11505229730/job/32028777115

This had a larger number of unsupported tests (106), however looking through these I think it is justifiable why these were marked as unsupported. 1 of these was marked as unsupported due to a negation in the requires, this is a simple change so will be opening a pr soon.

26 tests failed. 19 of these failures are covered by the draft pr's mentioned previously. 3 of these are compression tests which require the zstd library, 2 of these have been addressed in recent commits (#15864 and #15865). and the last 2 are part of #15816 and #15767.

9 tests unexpectedly passed, these should be marked as XFAIL: run-mode, will be doing that in a pr shortly

ayylol avatar Oct 25 '24 17:10 ayylol

Here is the results of the latest dry-run, after removing the REQUIRES/UNSUPPORTED logic for the triples: https://github.com/intel/llvm/actions/runs/11596214432

In total there were 63 Failures. 6 are addressed by #15789, another 9 are addressed by #15787. This leaves 48 tests that would need changes. 41 of these tests would need to be marked with REQUIRES: run-mode:

  • 20 are specifically meant for cuda or hip and cannot be built for the spir64 triple
  • 9 are tests only meant for windows
  • 6 are tests marked as TEMPORARY_DISABLED
  • 5 tests require Vulkan
  • 1 tests requires that zstd is not present in the environment.

The remaining 7 are fixed in a different manner:

  • 4 tests require lines be wrapped in %if run-mode, similar to #15787
  • 3 tests have lines that execute the binary without %{run} or %{run-unfiltered-devices}, similar to #15636

10 tests are marked as XPASS, these should be addressed by #15891

ayylol avatar Oct 30 '24 19:10 ayylol