v2022-12 tag is broken

[jakevdp]

The v2022-12 tag is broken due to the bug fixed in https://github.com/data-apis/array-api/pull/455, which as far as I can tell has not yet been made part of a release or tag.

The error can be seen here: https://github.com/google/jax/actions/runs/5073141741/jobs/9111776151?pr=16099

$ Run pytest --ci array-api-tests/array_api_tests/
ImportError while loading conftest '/home/runner/work/jax/jax/array-api-tests/conftest.py'.
array-api-tests/conftest.py:9: in <module>
    from array_api_tests import _array_module as xp
array-api-tests/array_api_tests/__init__.py:8: in <module>
    from ._array_module import mod as _xp
array-api-tests/array_api_tests/_array_module.py:4: in <module>
    from . import stubs
array-api-tests/array_api_tests/stubs.py:31: in <module>
    name_to_mod[name] = import_module(f"signatures.{name}")
E     File "/home/runner/work/jax/jax/array-api-tests/array-api/spec/API_specification/signatures/linalg.py", line 44[7](https://github.com/google/jax/actions/runs/5073141741/jobs/9111776151?pr=16099#step:6:8)
E       """
E       ^^^
E   SyntaxError: invalid escape sequence '\*'

This is a github action on a JAX PR implementing the array API: https://github.com/google/jax/pull/16099

Let me know if you have suggestions for how to properly configure the github action in order to pull-in the fix. Thanks!

[asmeurer]

This should be easy enough to fix (just make the docstrings raw).

Although somewhat related to this, I wonder if we should get rid of the tags here? We now have all versions of the spec in the main branch of this repo, exactly so that we can fix little issues like this.

I guess when this is fixed we'll also need to update the submodule in the test suite @honno

[jakevdp]

True it's easy enough to fix locally by manually editing the problematic files. I'm looking for suggestions of the best way to get this working in my github actions job (configured here: https://github.com/google/jax/blob/26fde03dccc595158f556e895d976b4aa3d41a75/.github/workflows/jax-array-api.yml)

[asmeurer]

There were still some bad escapes in main which I've fixed at https://github.com/data-apis/array-api/pull/632.

For array-api-tests, I think actually @honno still needs to update it to run against the main of this repo which has all the spec versions instead of the tags. So for now you'll need to work around it (a straightforward way is to manually compile the signatures with python -m compileall so that they aren't compiled with the syntax warnings turned into errors).

I'm going to leave this issue open because as I noted, we should figure out if we actually want to keep the tags on this repo.

[jakevdp]

I'm using sed as a workaround for now: https://github.com/google/jax/pull/16099/commits/c55b9e7d9fcc3a483b1501e446780bf2563f5802

[kgryte]

@honno Has this been resolved or does more work remain?

[honno]

Has this been resolved or does more work remain?

Haven't touched this yet (specifically writing a note in the README on tags and what they do/don't mean), need to find half an hour to sit down and do it :facepalm:

[honno]

The issue @jakevdp had should be resolved by https://github.com/data-apis/array-api-tests/pull/213 now.

we should figure out if we actually want to keep the tags on this repo.

@asmeurer If this is still interesting, could we close this and open a new issue? For 2023 we still kept with tags, which IMO makes sense.

[asmeurer]

I guess "tagging at release" is probably fine, but we should be very clear that those tags shouldn't really be used, and are only there for historical context. You should always use whatever is in main, because all versions of the standard are there and things like typo fixes are backported.

For the test suite, we should pin to a commit from main, but update it from time to time. Maybe we could look at something like dependabot to do this. Or we could just do it manually like we do now.