BUG: overridable functions from NEP 35 do not show up in `ARRAY_FUNCTIONS`

[neutrinoceros]

Describe the issue:

Functions that were made dispatchable via NEP 35 (which added the like keyword argument) do not appear to be so using the np.testing.overrides API.

Taking for instance np.asarray, whose like argument is documented as

If an array-like passed in as like supports the __array_function__ protocol, the result will be defined by it

Meanwhile, np.testing.overrides.get_overridable_numpy_array_functions's docstring says

List all numpy functions overridable via __array_function__

Since there's an API to check NEP 18 support, there should probably be one for NEP 35 too. I'm not sure wether the best route would be to extend the existing API to cover NEP 35 functions too, or if another set of functions should be added to np.testing.overrides.

Reproduce the code example:

import numpy as np
assert np.asarray in np.testing.overrides.get_overridable_numpy_array_functions()

Error message:

Traceback (most recent call last):
  File "/Users/clm/dev/astropy-project/coordinated/astropy/t.py", line 2, in <module>
    assert np.asarray in np.testing.overrides.get_overridable_numpy_array_functions()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Python and NumPy Versions:

2.1.1
3.12.6 (main, Sep  9 2024, 21:36:32) [Clang 18.1.8 ]

Runtime Environment:

[{'numpy_version': '2.1.1',
  'python': '3.12.6 (main, Sep  9 2024, 21:36:32) [Clang 18.1.8 ]',
  'uname': uname_result(system='Darwin', node='kwanzaabot.home', release='23.6.0', version='Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:52 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6020', machine='arm64')},
 {'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
                      'found': ['ASIMDHP'],
                      'not_found': ['ASIMDFHM']}}]

Context for the issue:

We started discussing supporting a subset of these functions in astropy.units (https://github.com/astropy/astropy/issues/17001), but I haven't found an easy systematic way to discover the whole set (in particular because we're already testing for implementation completeness of NEP 18 functions).

[seberg]

Thanks for the note. This should not be very hard, I think. The like= argument for (most, I think not all) is handled in C. They are thus slightly different (not wrapped, e.g. also don't have _implementation).

The solution may just be to add them manually (maybe with a decorator), and I suppose also add a test for that. Where the test may be the more interesting part even. This may not be quite trivial, but should be possible to fiugre out with some pointers, I suspect.

[neutrinoceros]

I can give it a try later this week.

[neutrinoceros]

Good news, it seems that many such functions are already decorated with @set_array_function_like_doc (ref gh-16935). Here are all 9 functions with this decorator

• require
• ones
• full
• fromfunction
• identity
• loadtxt
• genfromtxt
• eye
• tri

And here are every other NEP 35 function I could find (reading through gh-16935), which I'm guessing are all C-implemented

• array
• asarray
• asanyarray
• ascontiguousarray
• asfortranarray
• empty
• zeros
• fromstring
• fromiter
• fromfile
• frombuffer
• arange

conveniently, this second subset still has their docstrings defined in Python and can be discovered by grepping for

    .replace(
        "${ARRAY_FUNCTION_LIKE}",
        array_function_like_doc,
    ))

(which is essentially an inlining of @set_array_function_like_doc)

I think it's possible to resolve the problem with just some plumbing in that decorator... at least for the Python-implemented part. For the C-implemented part, I'm not sure what to do. Manually writing down the list of 12 functions somewhere is clearly the easy way out, but it's also not easily discoverable or maintainable. Adding a dedicated flag to numpy.lib.add_newdoc would provide a systematic way to do it but I generally don't like stacking argument to existing functions, even semi-private ones. How about tweaking add_newdoc so it automatically performs the str.replace and registers the function upon dedecting the substring "${ARRAY_FUNCTION_LIKE}" ?

[seberg]

Not sure that modifying add_newdoc is worth it, maybe we just add it as another line in the same place? But if you think it is, also fine.

[neutrinoceros]

@seberg I opened gh-27490 with my initial approach. Let me know what you think !