pyproj icon indicating copy to clipboard operation
pyproj copied to clipboard

Remove special-case casting support for numpy<2.0

Open greglucas opened this issue 2 years ago • 1 comments

Once we support only Numpy 2.0+ we can remove the casting checks we have added to the Cython code for single-point transforms and Geod's fwd/inv single-point as well. Right now we make sure to throw a TypeError so we get the right conversions, but Numpy 2.0+ will do this for us automatically.

https://github.com/pyproj4/pyproj/blob/f235f9b3067e4817ec34a95cfd1717d58fc95293/pyproj/_transformer.pyx#L768-L770

greglucas avatar Jun 22 '23 21:06 greglucas

Possibly helpful guide to follow on numpy support window: https://scientific-python.org/specs/spec-0000/

snowman2 avatar Jun 30 '23 14:06 snowman2

So could this work be done for the September 2025 release (3.8.0) of pyproj?

djhoese avatar Aug 12 '25 18:08 djhoese

So could this work be done for the September 2025 release (3.8.0) of pyproj?

It appears that it is time according to the schedule of support. I would support making the change.

snowman2 avatar Aug 12 '25 19:08 snowman2

@greglucas Is the correct fix here to remove all if isinstance: TypeError lines from:

https://github.com/pyproj4/pyproj/blob/878615decadbde676f9acc82bce9763d62fbacbe/pyproj/_transformer.pyx#L750-L764

So the TODO to resolve this issue is:

  1. Remove the above mentioned lines
  2. Remove the comment about type-checking
  3. Set 2.0 limit for numpy in dependencies in pyproject.toml
  4. Optional: Set C Macros for a target numpy version and/or the numpy deprecated api macro

Right?

djhoese avatar Aug 13 '25 15:08 djhoese

@djhoese, thanks for taking this on! It has been a while since I touched that, but I think I agree with that list of items to update. Make sure to do it for the geod fwd/inv functions too. Ping me for a review on any PRs and I can try and test with Cartopy too to make sure we are getting the behaviors we were looking for over there where I was trying to optimize the single-point transforms for that project.

greglucas avatar Aug 13 '25 15:08 greglucas

Thanks @greglucas. Rewording this, if we remove these checks, we expect numpy to raise a TypeError when trying to use the array as a scalar, right? Would it maybe be "cleaner" to just check the scalar-ness of the data instead of a try/except TypeError?

djhoese avatar Aug 13 '25 15:08 djhoese

if we remove these checks, we expect numpy to raise a TypeError when trying to use the array as a scalar, right?

I believe that is correct in numpy 2.0, but testing it now I'm not so sure... I still only get a Deprecation warning saying "in the future".

>>> float(np.array([1.]))
<stdin>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)

Would it maybe be "cleaner" to just check the scalar-ness of the data instead of a try/except TypeError?

I think this had to do with performance. The try/except was faster, but I'm sure we didn't test all combinations of options for narrowing the types allowed.

Here is the PR https://github.com/pyproj4/pyproj/pull/1204 with a few comments/discussions embedded within for historical reference. https://github.com/pyproj4/pyproj/pull/1204/files#r1052895488

greglucas avatar Aug 13 '25 15:08 greglucas

Set 2.0 limit for numpy in dependencies in pyproject.toml

Numpy isn't a dependency of pyproj. Will need lots of warnings in the documentation instead.

snowman2 avatar Aug 13 '25 16:08 snowman2

Well...shoot. That does complicate things doesn't it.

djhoese avatar Aug 13 '25 17:08 djhoese

I am thinking this may be better to do on a major release due to the backwards incompatible breaking nature of this change. May also not be bad to give it another year unless there is a pressing need.

snowman2 avatar Oct 03 '25 18:10 snowman2

Maybe a review of upcoming expected changes and split them up in one or two more releases? There's the PROJ 9.7.0 release, then what's next? It might be nice to have the PR ready and rebase it repeatedly to make sure it functions as expected over time then after the last non-major release we merge it and let it sit to be tested. Oh or a long release candidate or beta on pypi? My other packages tend to have an unstable CI environment that should pull pyproj in from github I think or maybe PyPI prerelease.

djhoese avatar Oct 04 '25 01:10 djhoese

I think a long release candidate makes sense. All good suggestions.

snowman2 avatar Oct 04 '25 02:10 snowman2

Yeah I'm mostly worried about needing to do a maintenance branch for the last non-major release which I always find annoying.

djhoese avatar Oct 04 '25 02:10 djhoese

Could do it the other way around and keep the main branch as the maintenance branch and a separate branch for the next major release with a running PR to the main branch as you suggested.

snowman2 avatar Oct 04 '25 02:10 snowman2