pyproj
pyproj copied to clipboard
Remove special-case casting support for numpy<2.0
Once we support only Numpy 2.0+ we can remove the casting checks we have added to the Cython code for single-point transforms and Geod's fwd/inv single-point as well. Right now we make sure to throw a TypeError so we get the right conversions, but Numpy 2.0+ will do this for us automatically.
https://github.com/pyproj4/pyproj/blob/f235f9b3067e4817ec34a95cfd1717d58fc95293/pyproj/_transformer.pyx#L768-L770
Possibly helpful guide to follow on numpy support window: https://scientific-python.org/specs/spec-0000/
So could this work be done for the September 2025 release (3.8.0) of pyproj?
So could this work be done for the September 2025 release (3.8.0) of pyproj?
It appears that it is time according to the schedule of support. I would support making the change.
@greglucas Is the correct fix here to remove all if isinstance: TypeError lines from:
https://github.com/pyproj4/pyproj/blob/878615decadbde676f9acc82bce9763d62fbacbe/pyproj/_transformer.pyx#L750-L764
So the TODO to resolve this issue is:
- Remove the above mentioned lines
- Remove the comment about type-checking
- Set 2.0 limit for numpy in dependencies in pyproject.toml
- Optional: Set C Macros for a target numpy version and/or the numpy deprecated api macro
Right?
@djhoese, thanks for taking this on! It has been a while since I touched that, but I think I agree with that list of items to update. Make sure to do it for the geod fwd/inv functions too. Ping me for a review on any PRs and I can try and test with Cartopy too to make sure we are getting the behaviors we were looking for over there where I was trying to optimize the single-point transforms for that project.
Thanks @greglucas. Rewording this, if we remove these checks, we expect numpy to raise a TypeError when trying to use the array as a scalar, right? Would it maybe be "cleaner" to just check the scalar-ness of the data instead of a try/except TypeError?
if we remove these checks, we expect numpy to raise a TypeError when trying to use the array as a scalar, right?
I believe that is correct in numpy 2.0, but testing it now I'm not so sure... I still only get a Deprecation warning saying "in the future".
>>> float(np.array([1.]))
<stdin>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
Would it maybe be "cleaner" to just check the scalar-ness of the data instead of a try/except TypeError?
I think this had to do with performance. The try/except was faster, but I'm sure we didn't test all combinations of options for narrowing the types allowed.
Here is the PR https://github.com/pyproj4/pyproj/pull/1204 with a few comments/discussions embedded within for historical reference. https://github.com/pyproj4/pyproj/pull/1204/files#r1052895488
Set 2.0 limit for numpy in dependencies in pyproject.toml
Numpy isn't a dependency of pyproj. Will need lots of warnings in the documentation instead.
Well...shoot. That does complicate things doesn't it.
I am thinking this may be better to do on a major release due to the backwards incompatible breaking nature of this change. May also not be bad to give it another year unless there is a pressing need.
Maybe a review of upcoming expected changes and split them up in one or two more releases? There's the PROJ 9.7.0 release, then what's next? It might be nice to have the PR ready and rebase it repeatedly to make sure it functions as expected over time then after the last non-major release we merge it and let it sit to be tested. Oh or a long release candidate or beta on pypi? My other packages tend to have an unstable CI environment that should pull pyproj in from github I think or maybe PyPI prerelease.
I think a long release candidate makes sense. All good suggestions.
Yeah I'm mostly worried about needing to do a maintenance branch for the last non-major release which I always find annoying.
Could do it the other way around and keep the main branch as the maintenance branch and a separate branch for the next major release with a running PR to the main branch as you suggested.