array-api-tests
array-api-tests copied to clipboard
Add simple example of running with different frameworks in the README
While the NumPy tests need to be run like so
import numpy.array_api as array_module
array_api
is not an attribute of other modules such as torch or tensorflow. Presumable these should be run like so:
import torch as array_module
import tensorflow as array_module
A couple of sentences stating this in the README might be helpful? Otherwise if they are not officially supported yet (I noticed there is only a github workflow for numpy), then maybe stating this in the README would be helpful? It isn't immediately obvious to me how I should run these tests for different backend frameworks. Any help appreciated!
It should work to use ARRAY_API_TESTS_MODULE=torch
, or import torch as array_module
if you are inserting the code in _array_module.py
.
Note that for torch and some other libraries that don't yet have very good compliance, you may have to write some simple wrappers to implement basic functions that are required for the test suite to work (e.g., no tests can run if asarray
isn't implemented).
Also worth noting there is still uncertainty for some libraries in what namespace they'll publicly adopt the spec with (e.g. top-level like pytorch
vs submodule like numpy.array_api
). Think an example for all adopters is a good idea though—sometime after a spec release and more stability regarding namespaces.
(Ideally entry points mitigates having to hard-document things, but for now that's opt-in. I imagine I'll add support for entry points for the test suite once a library other than NumPy adopts it.)
Thanks both for these very helpful comments. This all makes sense. Yes I agree this maybe becomes more important once more frameworks have better compliance. I'll leave this open for now but feel free to close.