python-neo icon indicating copy to clipboard operation
python-neo copied to clipboard

Support for NumPy 2.0 in Neo Core

Open zm711 opened this issue 1 year ago • 12 comments

following a policy of NEP 29 + 1 year.

zm711 avatar Jun 18 '24 13:06 zm711

Looks like we have some work to do :)

zm711 avatar Jun 18 '24 13:06 zm711

note that quantities 0.16.0, with support for NumPy 2.0, has just been released

apdavison avatar Aug 27 '24 11:08 apdavison

I updated the branch so we should see the neo problems vs the quantities in the testing!

zm711 avatar Aug 27 '24 12:08 zm711

I think @JuliaSprenger saw this error before right?

AssertionError: array(99.) * mV == array(99.) * mV

Actually this is an example of an error related to the fact we no longer have the copy behavior. this test was an assertNotEqual, but because the data is now a copy we edited in place and so the array is the same when it shouldn't be.

zm711 avatar Aug 27 '24 12:08 zm711

Let's squash this one when it is ready. A lot of this is me tweaking the action and then messing up on one version. So a lot of these are trash commits.

zm711 avatar Oct 17 '24 16:10 zm711

Okay current core failures can be reproduced with the following:

>>> import quantities as pq
>>> import numpy as np
>>> times = np.arange(10, dtype=np.float32)
>>> times.dtype
'float32'
>>> times= times * pq.ms
>>> times.dtype
'float64'

and for the other failing test

>>> import quantities as pq
>>> import numpy as np
>>> times = [1,2,3] * pq.s
>>> np.concatenate((times, times))
array([1,2,3,1,2,3]) * dimensionless

I need to read more about numpy 2.0 changes to figure out these parts of the tests that have changed. Any ideas @apdavison and @samuelgarcia ?

The first issue seems like it could be numpy but the second issue seems like quantities needs to hand np.concatenate better no?

zm711 avatar Oct 17 '24 18:10 zm711

Actually it was a change in default numpy behavior which requires an extra as type in tests. My bad.

zm711 avatar Oct 17 '24 22:10 zm711

Once #1585 is in main we can merge main into this which fixes the RTD error. Then this ready.

Note this is only for the core of Neo. I want to do the RawIO/IO in a separate PR.

zm711 avatar Oct 18 '24 11:10 zm711

Also a reminder please squash before merge and let's at minimum wait until after the point release :)

zm711 avatar Oct 18 '24 11:10 zm711

RTD is not building the correct version of neo. I don't know why. Maybe I just need to give it a little time (I had hoped switching to 3.12 would flush any cache, but it still had the wrong Neo and it had other errors). I'll just try to rerun RTD build this weekend and see if time fixes it.

zm711 avatar Oct 18 '24 14:10 zm711

Sam and I looked over the RTD failure and we aren't sure why it is happening. I think it should fix itself once this is merged into main since the problem showing up is fixed in main.

zm711 avatar Oct 21 '24 14:10 zm711

@apdavison this is ready for review, the RTD is broken with some cache issue, but I fixed the problem on main (and merging main into this branch didn't fix it). so our options are you review and merge ignoring the RTD problem or I open a new PR built off of the current main and make the exact same fixes to make sure all tests pass. What's your preference?

zm711 avatar Oct 28 '24 13:10 zm711

thanks for for @zm711 @apdavison and @alejoe91

I did not folow so much. Does finally we are making a copy of the buffer every time we create a data object from numpy or not ?

samuelgarcia avatar Jan 06 '25 14:01 samuelgarcia

It should force the user to return a view rather than a copy. So if someone wants to change the dtype at initialization they can't create a copy anymore at that stage. So they have to return the view and then change the dtype themselves (which will make the copy later). Same is true for quantity units. They can no longer change units at initialization. They have to do it later and make a copy at that time.

I think a couple structures may need to copy the view at a few places because of the views, but globally everything should start out as views.

zm711 avatar Jan 06 '25 14:01 zm711