array-api-tests icon indicating copy to clipboard operation
array-api-tests copied to clipboard

Update libraries we test on CI

Open honno opened this issue 1 year ago • 1 comments

  • Firstly we really need to
    • [x] Replace numpy.array_api with array-api-strict
      • numpy.array_api is going to be removed and supplanted by this third-party library
      • As bought up in https://github.com/data-apis/array-api-tests/pull/101#issuecomment-1949382387
    • [x] Also cover meta/ tests (we never did before!)
      • Would require one or two fixes of those tests.
    • [ ] Cover different ARRAY_API_TESTS_VERSION values too
      • https://github.com/data-apis/array-api-strict/issues/8 would be handy here
  • Then ideally we're testing libraries covered by array-api-compat
    • [ ] NumPy
    • [ ] PyTorch
    • [ ] Dask
    • CuPy is probably a no-go because GPUs on CI is too finnicky
  • I think it'd be really nice to test jax.experimental.array_api too as it has some distinct properties that we want to be mindful with for future test suite development.

honno avatar Feb 20 '24 17:02 honno

For array-api-compat I have a script that I can run on a CUDA machine that runs the CuPy tests. We could have something like that here. We could also do that for PyTorch GPU. We would then just have to run it ourselves manually from time to time.

asmeurer avatar Mar 21 '24 18:03 asmeurer