array-api-tests icon indicating copy to clipboard operation
array-api-tests copied to clipboard

Tracking remaining `2022.12` test suite coverage

Open honno opened this issue 2 years ago • 2 comments

The test suite has excellent coverage of the first 2021.12 spec release, but hasn't been updated to cover all the 2022.12 additions and changes. The following is a list of things that ideally the test suite would do:

  • [x] https://github.com/data-apis/array-api-tests/issues/187
    • Ideally we'd have a stubs package to use instead, i.e. https://github.com/data-apis/array-api/pull/472
    • [ ] Then we can support testing new special cases
  • [x] Test the fft extension
    • We definitely want to at least test the dtype and broadcasting behaviour of these functions—and by virtue, smoke test them with valid inputs. Values testing is non-trivial enough to probably not be worth doing.

honno avatar Jul 24 '23 18:07 honno

Ofc @asmeurer and folks feel free to edit the original post, this is just what immediately came to mind.

honno avatar Jul 24 '23 18:07 honno

The submodule thing also prevents the signature tests for working with any signature updates post 2022.12

We'll need to check with how the scipy stuff is going, but it's likely we'll want to prioritize fft testing, so that we can implement it numpy.array_api (https://github.com/numpy/numpy/issues/23892) and the compat library.

asmeurer avatar Jul 24 '23 18:07 asmeurer