tskit icon indicating copy to clipboard operation
tskit copied to clipboard

Numpy 2.0 Python

Open jeromekelleher opened this issue 1 year ago • 5 comments

Numpy 2.0 is about to drop: https://github.com/numpy/numpy/issues/24300

There is a ruff plugin which can check this as lint

This might be a motivation to switch to ruff

Would anyone like to try installing numpy 2.0 and see how the experience goes with running the test suite? Hopefully the Python level changes are small. See also #2893

jeromekelleher avatar Jan 23 '24 09:01 jeromekelleher

This is going to become an issue soon - we need to either verify that we work OK with numpy v2 or release a version that pins to numpy<2.

I've had a quick look using numpy 2.0.0rc2 and hit some issues.

jeromekelleher avatar May 20 '24 14:05 jeromekelleher

I've had a bit of a fiddle with this, and I've not been able to successfully build and import _tskit or the lwt_module (in the example_c_module). There's some funkiness going on with the build-time requirements and the updated numpy C API.

I don' think there's any fundamental problem here, but until we have numpy 2.0 packages easily installable across a range of platforms I don't think there's any point in trying to nail it down.

So, I vote that we push out an updated version of tskit that specify numpy<2 that should at least avoid some initial breakage when the new version drops. I guess we should do the same for msprime, but hopefully the tskit pin should be enough?

Any thoughts @benjeffery?

jeromekelleher avatar May 20 '24 15:05 jeromekelleher

I've had a go and got the module to import - the test suite is a whole other problem though with the chicken-and-egg msprime issues.

I am concerned that when we do switch our wheels built on numpy2 won't work if the user has numpy1 - and with many packages specifying numpy<2 we're going to cause conflicts.

As you suggest, best to just pin for now.

benjeffery avatar May 20 '24 15:05 benjeffery

This transition will be painful for a while. I know that fwdpy11 and moments are both pinning to <2 for now. It is hard to coordinate relaxing that, and there will also be issues with scipy/numba/etc..

molpopgen avatar May 20 '24 16:05 molpopgen

Conda forge packages seem to have gone through ok, so that might be a way forward in terms of CI anyway

jeromekelleher avatar May 20 '24 18:05 jeromekelleher