linopy icon indicating copy to clipboard operation
linopy copied to clipboard

Remove numpy pin

Open glatterf42 opened this issue 1 year ago • 3 comments

Gurobipy v11.0.3 was released on July 16 and seems to be compatible with numpy >= 2.0.0. At least locally, running the tests via pytest . and an editable install from this branch with numpy 2.0.1 and gurobipy 11.0.3, all tests are passing :)

The only thing I can't quite make sense of is this warning:

test/test_io.py::test_model_to_netcdf
  <frozen importlib._bootstrap>:488: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

glatterf42 avatar Jul 24 '24 08:07 glatterf42

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.98%. Comparing base (4735725) to head (e848cf1). Report is 14 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #332      +/-   ##
==========================================
- Coverage   86.00%   84.98%   -1.03%     
==========================================
  Files          17       17              
  Lines        4209     4209              
  Branches      989      989              
==========================================
- Hits         3620     3577      -43     
- Misses        430      473      +43     
  Partials      159      159              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jul 24 '24 08:07 codecov[bot]

Some tests on Windows also fail. This is not urgent, but feel free to dig in. Numpy deprecation with NPY201 is already being applied.

lkstrp avatar Jul 24 '24 09:07 lkstrp

Not what I expected, and I'm not using Windows. Comparing the latest py3.12 runs on Windows and Ubuntu, the only difference in the packages I notice is colorama 0.4.6 being installed on Windows, but I don't think that's the issue. Comparing to the successful Windows latest py3.10 run, the version of numpy differs. Numpy 2.0.1 is only used on Python 3.11 or greater, it seems.

My guess is that there's some kind of incompatibility with numpy still, but I'm struggling to troubleshoot this since 2.0.1 works fine on Ubuntu and Macos. The only differences in the workflows I notice are the Install <OS> dependency steps and those are very difficult to compare. Apart from that, the python version, package version, etc, are all identical. Looking at the traceback, the only package involved seems to be polars and they do actually have an open issue with the title sounding very similar to the error here: https://github.com/pola-rs/polars/issues/16863 Unfortunately, I'm not sure if this is all that useful. The proposed solution is to not used concat() in favor of a little refactoring, but I'm not sure you can apply the same here. As far as I can tell from quickly looking at polar's source, I can't find a special treatment of Windows for the functions that seem to cause the error.

Happy to help further, but I'm slowly running out of ideas. Maybe it's still best to hold off on removing the numpy pin for now and hope the responsible party releases a fix soon. Just a bit inconvenient for my special case: I only want to use Highs with linopy (for now), which works well with numpy 2.x, as does our side of packages, so my solution for now is to use my own branch which already allows numpy 2.x and just be aware that gurobipy can't be used with that (and that I'm anyway not on Windows, it seems).

glatterf42 avatar Jul 24 '24 10:07 glatterf42

To prevent windows from blocking this change, platform specifier could be used for numpy?

numpy>=2; sys_platform != 'win32'  # or just numpy without version specifier
numpy<2; sys_platform == 'win32'

ollie-bell avatar Nov 14 '24 10:11 ollie-bell

@ollie-bell I think that is a good idea. applied it. @glatterf42 sorry for the mess, was trying to do it on GH with suggestions first...

FabianHofmann avatar Nov 14 '24 13:11 FabianHofmann

@FabianHofmann @glatterf42 is any help wanted with this? Happy to try and pick it up if so

ollie-bell avatar Feb 04 '25 20:02 ollie-bell

@FabianHofmann @glatterf42 is any help wanted with this? Happy to try and pick it up if so

hey @ollie-bell, yes that would be wonderful!

FabianHofmann avatar Feb 04 '25 22:02 FabianHofmann

Closing, since it was already merged as part of #415

coroa avatar Mar 12 '25 08:03 coroa