HiGHS icon indicating copy to clipboard operation
HiGHS copied to clipboard

Make HiGHS installable via PyPI

Open l-kotzur opened this issue 2 years ago • 38 comments

I would be great if one could call pip install highs. This would make the solver in general much easier installable and accessible.

@michaelbynum wrote already an appsi wrapper from Pyomo to HiGHS, as pointed in Pyomo/pyomo#2094. Unfortunately, this has stalled and has not been merged to the main branch. One of the main reasons is that HiGHS is not installable via a package manager. Therefore, it cannot be integrated into a proper CI-pipeline, I guess.

@mckib2 wrote wrappers for sckit and for scipy #156 and brought older versions to PyPI. Would it be possible to make HiGHs as an own PyPI package such that Pyomo and others can more easily use it?

Unfortunately, I personally do not have any experience in packaging C++ libraries.

l-kotzur avatar Jul 27 '22 15:07 l-kotzur

@mckib2 wrote wrappers for sckit and for scipy #156 and brought older versions to PyPI. Would it be possible to make HiGHs as an own PyPI package such that Pyomo and others can more easily use it?

The old scikit that you linked to was a proof-of-concept for integrating into the SciPy build system. All that work has been integrated into SciPy and improved upon with the MIP solver accessible in the upcoming 1.9.0 release. Instead of the old buildutils build system, we now use meson. We expect to develop an object oriented interface in the coming months. 1.9.0 should also contain a very recent version of HiGHS with SciPy binaries available across many platforms and installable via pip.

Any reason you can't use/build on SciPy's linprog function? I believe cvxpy is taking this approach and you'll get support for Python specific build and usage issues

mckib2 avatar Jul 27 '22 16:07 mckib2

Any reason you can't use/build on SciPy's linprog function? I believe cvxpy is taking this approach and you'll get support for Python specific build and usage issues

Thanks for the pointer on this @mckib2 We need HiGHS to be installable via a package manager for a number of projects. @michaelbynum and @galabovaa have managed this for MacOS and Linux, but Windows is proving to be very tricky.

I don't know whether we can use/build on SciPy's linprog function, but @galabovaa is soon back from holiday and will have a view

jajhall avatar Jul 27 '22 21:07 jajhall

@jajhall Only supporting Linux and MacOS would be fine at least for me. Coin-CBC is only supported on Linux and MacOS as well atm, if I am not wrong.

l-kotzur avatar Aug 25 '22 07:08 l-kotzur

For the prototype "thin" Python wrapper highspy around the C++ class, pip install highspy works on two Ubuntu installations, and is not known to fail on any other Linux installation.

jajhall avatar Sep 11 '22 21:09 jajhall

@michaelbynum Would this be sufficient to integrate the appsi-HiGHS interface to pyomo?

l-kotzur avatar Sep 23 '22 15:09 l-kotzur

pip install highspy also works on (at least one) MacOS. Windows is WIP.

jajhall avatar Sep 23 '22 15:09 jajhall

Having highspy on PyPI is great! It helped me a lot in adding an interface for HiGHS in qpsolvers.

Looking forward to the upcoming release (since currently 1.1.2.dev3 is pre-release) :smiley:

Also, not part of this issue but likely a follow-up feature request, I'll be looking forward for the addition of get/setHotStart to the interface (keeping track in https://github.com/stephane-caron/qpsolvers/issues/94).

stephane-caron avatar Sep 29 '22 08:09 stephane-caron

Also, not part of this issue but likely a follow-up feature request, I'll be looking forward for the addition of get/setHotStart to the interface (keeping track in stephane-caron/qpsolvers#94).

Hot starting the QP solver is not possible at present, and might not be possible for a long time. My other comments relate to hot-starting the LP solver

jajhall avatar Sep 29 '22 08:09 jajhall

I could not find highspy on PyPI. I know pip install works locally, but is it on PyPI yet? If so, could anyone share a link?

@l-kotzur, yes - having highspy on PyPI would be sufficient to integrate the appsi-HiGHS interface into Pyomo.

michaelbynum avatar Sep 29 '22 13:09 michaelbynum

You can find it at https://pypi.org/project/highspy/. It does not appear in search because it is still a pre-release.

stephane-caron avatar Sep 29 '22 13:09 stephane-caron

Thanks!

michaelbynum avatar Sep 29 '22 13:09 michaelbynum

Hi, do you know when/if there were plans for the pip installed version to work on Windows? Thanks so much.

jackbrucesimpson avatar Nov 02 '22 03:11 jackbrucesimpson

We certainly "plan" to get pip install highspy to work on Windows, but achieving it is another matter!

jajhall avatar Nov 02 '22 07:11 jajhall

@l-kotzur, The Pyomo-Appsi interface to HiGHS is now in Pyomo main and will be in the next release.

michaelbynum avatar Nov 02 '22 13:11 michaelbynum

@l-kotzur, The Pyomo-Appsi interface to HiGHS is now in Pyomo main and will be in the next release.

@michaelbynum I have seen it. Those are great news, thanks. I am looking forward to test it.

l-kotzur avatar Nov 02 '22 13:11 l-kotzur

Hello @jajhall, what is the issue to pip install on windows? I would be glad to help if you think I could. We develop on windows where I work.

The HiGHS Rust interface works on Windows when we install cmake and llvm. But I imagine it is more challenging with PyPI?

trigaut avatar Nov 18 '22 07:11 trigaut

Hello @jajhall, what is the issue to pip install on windows? I would be glad to help if you think I could. We develop on windows where I work.

I don't know specifically, because it's being worked on by @galabovaa. Currently she's combining work on this with integrating HiGHS into another major solver package. Perhaps there's some specific issue that you could help her with, but I'll leave her to ask. Many thanks!

jajhall avatar Nov 18 '22 10:11 jajhall

Hi @trigaut

Currently on Windows, pip install seems to be generating a wheel, however, when I call "import highspy" from python I get a "DLL not found" error.

I am not an expert on Windows and I have been struggling to get the paths correct.

Any help would be much appreciated. You could have a look at the win branch which contains some minor changes to the CMake files.

Thanks, Ivet

galabovaa avatar Nov 21 '22 13:11 galabovaa

@trigaut: It's worth (re-?)emphasising that pip install highspy works on Linux and MacOS

jajhall avatar Nov 21 '22 14:11 jajhall

I just managed to build a wheel for Windows 64bit cp39: https://pypi.org/project/highspy/

If you would please try to install from your windows machines and test if it works on your side that would be great!

galabovaa avatar Nov 21 '22 14:11 galabovaa

The example from the README is working for my case on Windows 10. Great support, thanks!

l-kotzur avatar Nov 21 '22 14:11 l-kotzur

Version 1.1.2.dev3 was working on Linux but when I just tested out updating to 1.4.0.dev0 it appears to fail.

[image: image.png]

On Tue, 22 Nov 2022 at 01:56, Leander Kotzur @.***> wrote:

The example from the README is working for my case on Windows 10. Great support, thanks!

— Reply to this email directly, view it on GitHub https://github.com/ERGO-Code/HiGHS/issues/925#issuecomment-1322189189, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQC6PK4N6HDQTKG5PL42UDWJOESRANCNFSM542DWKZA . You are receiving this because you commented.Message ID: @.***>

jackbrucesimpson avatar Nov 22 '22 00:11 jackbrucesimpson

Version 1.1.2.dev3 was working on Linux but when I just tested out updating to 1.4.0.dev0 it appears to fail.

1.4.0.dev0 doesn't have a recent branch from @galabovaa merged into it

jajhall avatar Nov 22 '22 09:11 jajhall

Yesterday I uploaded highspy 1.4.0 only for Windows so trying to update on Linux would give users an error. I will update linux and macOS as well to 1.4.0 once we merge the latest changes into master.

galabovaa avatar Nov 22 '22 09:11 galabovaa

Thanks so much!

On Tue, 22 Nov 2022 at 20:54, galabovaa @.***> wrote:

Yesterday I uploaded highspy 1.4.0 only for Windows so trying to update on Linux would give users an error. I will update linux and macOS as well to 1.4.0 once we merge the latest changes into master.

— Reply to this email directly, view it on GitHub https://github.com/ERGO-Code/HiGHS/issues/925#issuecomment-1323394499, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQC6PLF5BI5FBKQ3MF6TXTWJSJ5LANCNFSM542DWKZA . You are receiving this because you commented.Message ID: @.***>

jackbrucesimpson avatar Nov 22 '22 11:11 jackbrucesimpson

Just tested pip install highspy and to solve many problems modeled with pyomo and using the "appsi_highs" SolverFactory. It works like a charm! Thanks !

trigaut avatar Nov 23 '22 11:11 trigaut

For me the same. Therefore also again many thanks to @michaelbynum and the whole HiGHS-team. For my instance, I had at least a factor three faster solving time in comparison to coin-cbc. Just waiting for the official Pyomo-release to set it as default solver.

l-kotzur avatar Nov 23 '22 11:11 l-kotzur

@galabovaa I've tested it on my Windows laptop, and it works perfectly!

jajhall avatar Nov 23 '22 11:11 jajhall

Excellent news! I would also like to thank @galabovaa for all the effort putting together the PyPI releases. It made things much easier for Pyomo.

michaelbynum avatar Nov 23 '22 14:11 michaelbynum

Is it planned to add a source distribution to the pypi release? This would simplify the creation of conda package.

julian-belina avatar Nov 29 '22 15:11 julian-belina