array-api-tests icon indicating copy to clipboard operation
array-api-tests copied to clipboard

Add simple example of running with different frameworks in the README

Open djl11 opened this issue 3 years ago • 3 comments

While the NumPy tests need to be run like so

import numpy.array_api as array_module

array_api is not an attribute of other modules such as torch or tensorflow. Presumable these should be run like so:

import torch as array_module

import tensorflow as array_module

A couple of sentences stating this in the README might be helpful? Otherwise if they are not officially supported yet (I noticed there is only a github workflow for numpy), then maybe stating this in the README would be helpful? It isn't immediately obvious to me how I should run these tests for different backend frameworks. Any help appreciated!

djl11 avatar Jan 25 '22 18:01 djl11

It should work to use ARRAY_API_TESTS_MODULE=torch, or import torch as array_module if you are inserting the code in _array_module.py.

Note that for torch and some other libraries that don't yet have very good compliance, you may have to write some simple wrappers to implement basic functions that are required for the test suite to work (e.g., no tests can run if asarray isn't implemented).

asmeurer avatar Jan 25 '22 20:01 asmeurer

Also worth noting there is still uncertainty for some libraries in what namespace they'll publicly adopt the spec with (e.g. top-level like pytorch vs submodule like numpy.array_api). Think an example for all adopters is a good idea though—sometime after a spec release and more stability regarding namespaces.

(Ideally entry points mitigates having to hard-document things, but for now that's opt-in. I imagine I'll add support for entry points for the test suite once a library other than NumPy adopts it.)

honno avatar Jan 25 '22 22:01 honno

Thanks both for these very helpful comments. This all makes sense. Yes I agree this maybe becomes more important once more frameworks have better compliance. I'll leave this open for now but feel free to close.

djl11 avatar Jan 26 '22 08:01 djl11