iree icon indicating copy to clipboard operation
iree copied to clipboard

Run `iree-import-onnx` test(s) on CI

Open ScottTodd opened this issue 1 year ago • 1 comments

https://github.com/openxla/iree/pull/16622 fixes an issue that should have been caught by presubmit tests.

We have a test but it has a soft failure mode if onnx is not installed: https://github.com/openxla/iree/blob/4c97ab6800820392589ce440f5b6b98f9bd855da/compiler/bindings/python/test/tools/import_onnx_test.py#L51-L56

I don't see a convenient way right now to either

  • install that dep in the existing job that runs the test (build_all or a test job like test_gpu, complicated due to Docker)
  • run the test in a job that can easily install it (pkgci, which doesn't use ctest)

This ONNX importer test/component should be low complexity and be unlikely to break (famous last words), so just putting this on the backlog to handle with future infrastructure work.

ScottTodd avatar Feb 29 '24 20:02 ScottTodd

We should also get https://github.com/iree-org/iree/blob/main/compiler/bindings/python/test/extras/fx_importer_test.py actually running by installing torch. That can be in a test workflow similar to https://github.com/iree-org/iree/blob/main/.github/workflows/pkgci_test_tensorflow_cpu.yml.

  • pkgci_test_torch_cpu.yml
  • merge into something like pkgci_test_frameworks.yml with multiple jobs/steps?
  • pkgci_test_onnx.yml (or onnx under pkgci_test_frameworks.yml) could have https://github.com/iree-org/iree-test-suites/tree/main/onnx_ops and the iree-import-onnx tests

ScottTodd avatar Aug 29 '24 22:08 ScottTodd