Support for Numpy 2.0

[sanurielf]

HI,

As reported in https://github.com/conda-forge/ta-lib-feedstock/pull/45 ta-lib builds are failing against future Numpy 2.0. Any plans to support this? If so we can use this issue to track the implementation.

[mrjbq7]

It would be nice. My understanding is that it requires Cython support first.

[sanurielf]

I took a look at the builds error, and seems it is a Cython problem as reported in https://github.com/cython/cython/issues/6249

talib/_ta_lib.c:10268:41: error: no member named 'subarray' in 'struct _PyArray_Descr' __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=967035&view=logs&j=8ca717d7-2f91-593f-2e86-7b14f42b0119&t=f56d028c-9d05-5be8-63d1-db69e7fd673f&l=1181

So, totally agree with you @mrjbq7 , we should wait for a Cython stable support.

[filbranden]

Just pointing out that building from the latest trunk I get a successful build that works with numpy 2.

I don't know how you plan to support numpy 2 vs 1 moving forward, I don't think it's possible to support both, you might want to do a somewhat larger version bump to indicate numpy 2 only versions moving forward. But I believe there's not really a blocker to releasing a numpy 2 version of TA-Lib at this point.

[mrjbq7]

I would imagine supporting numpy1 is something people might want for some time, what are other projects doing? Building with the compat layer?

[filbranden]

I'm not aware of any compatibility layer for Numpy C API

For example https://github.com/explosion/spaCy released v3.8.0 with numpy 2 support, while v3.7.x has numpy 1 support, so bumping the version in a way that would allow easily pinning (e.g. <0.5 vs >=0.5) would probably be helpful

[mrjbq7]

The Numpy 2.0 Migration Guide mentions the idea of including npy2_compat.h when compiling against numpy 1, and I saw reference to a #define that would also force compatibility with an older API but I haven't looked recently.

Anyway, given how slowly some places update libraries, it makes me wonder if having a hard version split against numpy 2 is a good idea or not...

[filbranden]

Ah I see on PyPI you're releasing only a tarball and not a wheel, so technically since you're distributing sources you might get compatibility with both numpy 1 and numpy 2

There's still a problem that whatever you specify on your dependencies goes, so once you have numpy 2 allowed there (by dropping the numpy<2 part) it will pick numpy 2 by default

But there's ways for those who want to enforce numpy 1 to keep that, e.g. by using pip install --no-build-isolation with a specific set of dependencies that include numpy 1 installed first

[filbranden]

Hi 👋

Any updates on support for numpy 2?

As pointed out, building from trunk against numpy 2 seems to work just fine

So it might just be a matter of updating the requirements to unpin numpy and allow numpy 2 and releasing a new version including that update

Thanks!

[mrjbq7]

I could do that but because we bundle pre generated C files, I’m concerned it will effectively pin to numpy 2.

Needs to be investigated more, sorry. I will get to it if someone else doesn’t!

[filbranden]

As a suggestion, since you seem to lean towards keeping numpy 1 compatibility around:

• Create a branch release-0.4 at the current point and make that the numpy 1 branch. In the future, if you need to release another fix for the numpy 1 version, push it to that branch and tag a 0.4.33, 0.4.34 there
• Have numpy 1 users specify TA-Lib<0.5 in their requirements file, that will always get the latest 0.4.x available
• Bump up the version to 0.5.0 or 0.5.1 etc. for the numpy version, you can keep using the master branch for that, no need to create a specific release branch until you decide to diverge again (on 0.6.x, or 1.0.x, etc.)

[mrjbq7]

That’s a good idea.

It’s not that I love numpy 1, I just think there are users that can’t upgrade yet for various reasons and I want to support that.

[filbranden]

Hi @mrjbq7 just checking if you've had the chance to look into starting a separate branch for numpy 2 support

Let me know if there's anything I can help with

My previous testing suggests that things build under numpy 2, but perhaps I'm missing some more thorough testing to ensure they actually work as expected

Thank you for your work on TA-Lib, much appreciated!

[jvrodrisan]

Hi all, @filbranden my testing also suggests that latest cython version (alpha) and numpy 2 build ta-lib successfully, however when ta-lib is imported it raises an exception around size of numpy dtype. Does that occur to you too?

If there is something I can do to help on this one, I will.

Thanks for all your work!

[mrjbq7]

Released ta-lib 0.5.0 with numpy 2.0 support.

Also released ta-lib 0.4.33 with numpy<2.0 support.